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Executive  Summary 


The  U.S.  Army  Corps  of  Engineers  (USAGE)  has  a  proud  tradition  of  producing 
technically  excellent  products  for  the  Army,  its  military  partners,  and  the  nation. 
Many  of  its  civil  works  and  military  construction  projects  are  recognized  as  tech¬ 
nically  advanced,  state-of-the-art  engineering  achievements. 

Because  it  wants  to  ensure  that  it  continues  to  deliver  quality  products  and  serv¬ 
ices  well  into  the  future,  the  Corps  has  instituted  a  number  of  significant  cultural, 
technical,  and  methodological  improvements  focusing  on  quality.  However,  it 
recognizes  that  it  needs  a  comprehensive  quality  management  system — a  system 
that  demonstrates  to  potential  customers  that  the  Corps  is  committed  to  providing 
quality  products  and  services  and  that  it  has  a  bona  fide  quality  management  sys¬ 
tem  in  place  to  meet  that  commitment.  The  Corps  concluded  that  the  best  tool  to 
achieve  those  goals  is  the  ISO  9000  quality  management  system. 

To  test  the  concept  of  operating  under  an  ISO  quality  management  system, 
USAGE  headquarters  selected  four  engineering  and  two  construction  divisions  to 
participate  in  a  pilot  program.  On  the  basis  of  an  assessment  of  the  pilot  program, 
LMI  concluded  that  ISO  9000  registration  offers  valuable  benefits  to  the  Corps. 
Pilot  organizations  reported  that  their  postregistration  operations  are  streamlined, 
more  efficient,  and  consistent  with  the  operation  of  an  engineering  and  construc¬ 
tion  organization.  The  rigor  of  maintaining  registration  is  of  value  as  well;  it 
forces  organizational  self-examination  in  preparation  for  external  audits.  Regis¬ 
tration  is  just  the  initial  step;  the  positive  effects  of  operating  under  an  ISO  9000 
quality  system  can  be  expected  to  continue  indefinitely,  not  just  immediately  after 
implementation. 

Less  tangible,  but  equally  worthy,  is  the  effect  of  ISO  9000  on  morale.  ISO  9000 
represents  a  positive  cultural  change.  The  divisions  have  found  that  operating  un¬ 
der  the  ISO  quality  management  system  has  improved  internal  communication, 
increased  teamwork,  improved  die  abiUty  of  individuals  to  see  the  big  picture,  re¬ 
duced  the  time  required  to  deal  with  routine  procedures,  clarified  responsibilities, 
and  reduced  errors  and  rework.  The  benefits  of  operating  under  the  ISO  quality 
management  system  are  undeniable.  Although  the  organizations  participating  in 


111 


the  pilot  program  were  enthusiastic  about  both  the  experience  and  the  results  of 
the  ISO  9000  certification  process,  some  viewed  the  time  and  cost  of  the  process 
as  too  high.  However,  the  Corps  can  take  advantage  of  the  lessons  learned  in  the 
pilot  program  to  improve  the  process.  Specifically,  to  facilitate  the  implementa¬ 
tion  of  ISO  9000  throughout  the  Corps,  we  recommend  that  USACE  take  the  fol¬ 
lowing  steps: 

♦  Adopt  ISO  9000  as  the  Corps  quality  management  standard,  revising  En¬ 
gineering  Regulation  1110-1-12  accordingly. 

♦  Direct  all  engineering  and  construction  divisions  to  achieve  and  maintain 
registration  to  the  ISO  9000  standard. 


♦ 

♦ 


Develop  customized  training  materials. 

Create  a  model  policy  and  procedures  manual  that  all  Corps  organizations 
can  use  as  a  starting  point  for  creating  their  own  manuals. 

♦  Form  a  headquarters  working  group  to  be  the  nerve  center  and  clearing¬ 
house  for  ISO  experience  within  the  Corps. 

♦  Develop  performance  metrics  and  continuous  improvement  measures  to 
track  progress  at  each  division  that  achieves  ISO  certification. 

♦  Establish  a  master  contract  with  a  single  ISO  registrar  to  reduce  the  time. 
and  cost  of  each  division’s  procuring  those  services  and  to  reap  the  ad¬ 
vantages  of  having  a  registrar  that  is  familiar  with  USACE  and  its  opera¬ 
tions. 

♦  Begin  concurrently  implementing  ISO  quality  management  systems  in 
multiple  divisions  within  a  district,  including  any  division  (such  as  pro¬ 
gramming  and  contracts)  involved  with  delivering  service  to  customers. 

♦  Test  application  of  ISO  9000  beyond  the  district  level. 

♦  Train  major  subordinate  conmiand  quality  assurance  people  in  ISO  inter¬ 
nal  auditing  procedures. 

♦  Apply  electronic  and  Web  technology  for  document  control. 

ISO  9000  is  a  tool  that  provides  a  framework  from  which  organizations  can  strive 
toward  continuous  improvement  and  achieve  excellence.  By  operating  under  a 
common  ISO  quality  management  system,  USACE  will  be  able  to  manage  inno¬ 
vation  effectively,  respond  appropriately  to  rapidly  changing  customer  require¬ 
ments,  continually  improve  its  processes,  and  gain  the  dedicated  involvement  of 
everyone  in  the  organization.  In  short,  it  can  become  an  organization  that  is  satis¬ 
fied  only  when  customer  requirements  are  always  met  and  service  expectations  are 
surpassed. 
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Chapter  1 

Introduction 


The  U.S.  Army  Corps  of  Engineers  (USAGE)  has  a  proud  tradition  of  producing 
technically  excellent  products  for  the  Army,  its  military  partners,  and  the  nation. 
Many  of  its  civil  works  and  military  construction  projects  are  recognized  as  tech¬ 
nically  advanced,  state-of-the  art  engineering  achievements.  However,  in  the  early 
1990s,  the  Corps  began  to  sense  that  its  customers  were  not  always  satisfied  with 
its  products.  After  investigating  the  reasons  for  this  growing  dissatisfaction,  the 
Corps  found  that  its  customers  now  want  more  than  the  traditional  focus  on  tech¬ 
nical  results — technical  sufficiency  is  expected.  USAGE  customers  want  their 
projects  delivered  on  time,  at  a  competitive  price,  and  they  want  excellent  service. 
In  short,  customers  are  demanding  a  higher  level  of  quality  in  the  engineering  and 
construction  management  services  they  are  purchasing  from  USAGE. 

The  Corps’  customers  (clients  and  partners)  are  becoming  increasingly  discrimi¬ 
nating  buyers,  largely  because  their  budgets  and  the  size  of  their  technical  staffs 
are  shrinking  and  their  schedules  are  compressed.  These  new  realities  have  caused 
the  Corps  to  reevaluate  both  its  culture  and  its  business  and  technical  methods  as 
it  strives  to  improve  its  overall  performance  today  and  into  the  future. 

USAGE  recognizes  that  its  future — to  be  the  world’s  premier  engineering  and 
construction  management  organization — depends  on  its  ability  to  deliver  quality 
products  and  services.  That  recognition  is  formalized  in  the  Corps’  strategic  vi¬ 
sion,  issued  in  1996  by  Lieutenant  General  Joe  N.  Ballard,  Commander,  USAGE, 
to  become  “a  vital  part  of  the  Army  ;  the  Engineer  team  of  choice — ^responding  to 
our  Nation’s  needs  in  peace  and  war;  a  values-based  organization — ^respected,  re¬ 
sponsive  and  reliable.” 

Moreover,  Corps  offices  throughout  the  nation  and  overseas  are  accepting  the 
concept  that  quality  is  delivered  when  its  customers — ^not  the  Corps — ^perceive 
that  it  has  been  delivered.  As  a  result  of  this  culture  change,  the  Corps  is  increas¬ 
ingly  finding  that  it  is  building  and  sustaining  better  customer  relationships. 

The  Corps  has  undertaken  a  number  of  initiatives  to  ensure  that  it  consistently  de¬ 
livers  quality  products  and  services — that  it  meets  customer  expectations.  It  is 
making  significant  progress  through  its  nationally  recognized  “partnering”  leader¬ 
ship,  first  with  its  construction  contractors,  then  with  its  architect-engineering 
(A-E)  firms,  and  now  with  its  customers.  Through  its  partnering  initiative,  the 
Corps  works  closely  with  all  parties  involved  in  a  project  to  identify  and  attain 
mutual  goals  and  objectives. 


1-1 


Other  initiatives  under  way  to  improve  the  quality  of  Corps  products  and  services 
include  the  following: 

♦  Soliciting  formal  customer  feedback  on  Corps  performance.  Many  local 
Corps  offices  have  done  this  in  the  past.  Corporate  emphasis  is  now  di¬ 
rected  at  asking  customers  to  provide  information  that  will  improve  the 
Corps’  service  at  all  levels  and  in  all  customer-service  areas.  And  most 
important,  the  Corps  is  emphasizing  effective  and  timely  follow-up  on  that 
information. 

♦  Capitalizing  on  the  availability  of  computer-aided  design  and  drafting 
(CADD)  technology.  Its  efforts  in  developing  CADD  standards,  generic 
details,  standard  project  designs,  design  discipline  analyses,  and  design- 
review  and  lesson-learned  tools  are  enabling  it  to  produce  even  better  en¬ 
gineering  results,  more  quickly  and  at  lower  cost. 

♦  Controlling  direct  engineering  costs,  departmental  overhead,  and  general 
and  administrative  overhead.  These  initiatives  are  already  reducing  the 
Corps’  design  costs  for  military  projects. 

♦  Streamlining  the  complex  processes  related  to  civil  works  projects.  The 
Corps  has  been  able  to  reduce  the  time  from  project  initiation  to  comple¬ 
tion  dramatically. 

♦  Developing  new  policy  and  guidance  to  improve  uniformity  and  timeliness 
and  to  enable  innovation  in  the  acquisition  of  A-E  and  construction  serv¬ 
ices. 

Although  it  has  instituted  significant  cultural,  technical,  and  methodological  im¬ 
provements  focusing  on  quality,  the  Corps  has  recognized  that  it  needs  a  compre¬ 
hensive  quality  management  system — a  system  that  will  enable  it  to  make  sure 
customers’  requirements  are  fully  understood  before  work  begins,  to  provide  cus¬ 
tomers  with  ongoing  feedback  during  product  development,  to  ensure  that  the 
finished  product  satisfies  the  requirements  established  and  agreed  to  by  both  par¬ 
ties,  and  to  provide  a  framework  for  systematically  evaluating  and  improving  its 
major  processes.  In  short,  the  Corps  needs  a  quality  management  system  that 
demonstrates  to  potential  customers  that  the  Corps  is  committed  to  providing 
quality  products  and  services  and  that  it  has  a  bona  fide  quality  management  sys¬ 
tem  in  place  to  meet  that  commitment.  USACE  senior  managers  believe  that  the 
best  tool  to  achieve  those  goals  is  registration  to  the  ISO  9000  quality  manage¬ 
ment  system. 
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Introduction 


Background 

The  ISO  9000  quality  system  was  developed  by  the  International  Organization  for 
Standardization  to  bring  uniformity  to  the  area  of  quality  assurance.'  Uniformity 
was  needed  because  the  number  of  quality  standards  being  used  throughout  the 
world  had  proliferated;  each  country — and  often  each  industry — ^had  instituted  its 
own  quality  standards  as  quality  became  more  important  to  consumers.  The  large 
number  of  different  quality  standards  made  compliance  with  them  problematic  for 
many  companies.  Not  only  was  it  difficult  and  costly  for  companies  to  keep  track 
of  the  wide  range  of  requirements  and  regulations  imposed  by  different  customers, 
multinational  firms  often  had  to  juggle  conflicting  regulations  or  face  the  fact  they 
might  not  be  able  to  sell  products  designed  for  one  country  in  another  because 
they  did  not  meet  that  country’s  unique  quaUty  assurance  standards. 

First  issued  in  1987  and  revised  in  1994,  the  ISO  quality  system  looks  at  the  pri¬ 
mary  elements  that  contribute  to  a  company’s  ability  to  provide  the  products  and 
services  that  meet  its  customers’  stated  needs.^  Those  elements  range  from  con¬ 
tract  reviews  with  the  customer,  inspection  and  testing  procedures,  and  employee 
training,  to  management’s  responsibility  for  and  commitment  to  quality. 

Companies  can  assure  their  customers  that  they  have  an  effective  quality  man¬ 
agement  system  by  proving  to  an  ISO  registrar  that  they  meet  all  applicable  ISO 
9000  requirements — ^that  they  have  developed  procedures  that  address  all  applica¬ 
ble  elements  and  that  they  use  them  consistently.  Third-party  audits  of  a  com¬ 
pany’s  quality  management  system — audits  conducted  by  independent  registrars 
whose  findings  are  acceptable  to  both  the  customer  and  the  supplier — are  the 
hallmark  of  ISO  9000. 

That  the  system  is  effective  has  been  reported  by  many  companies  that  have  un¬ 
dertaken  the  ISO  9000  certification  process.  In  addition  to  satisfying  their  custom¬ 
ers,  companies  operating  under  ISO  9000  say  they  have  improved  productivity 
and  reduced  the  costs  associated  with  inefficient  operations  and  wasted  efforts. 
ISO  9000  ensures  that  the  organization  does  the  right  things  right,  the  first  time 
and  all  the  time.  Among  the  many  significant  benefits  that  are  widely  reported  are 

♦  better  understanding  of  customer  needs; 


*  The  lOS  was  founded  shortly  after  the  end  of  World  War  n  to  set  uniform  standards  for 
products  and  thus  advance  international  commerce.  Since  its  founding,  the  lOS  has  issued  more 
than  8,000  standards  and  technical  reports.  Each  standard  is  assigned  a  numerical  designation.  The 
prefix — ^ISO,  fi'om  the  Greek  isos  meaning  “equal” — ^was  chosen  to  indicate  that  the  standards 
would  apply  to  all  users  equally,  regardless  of  a  company’s  size,  its  products,  its  services,  or  the 
country  in  which  it  is  located. 

^  ISO  standards  are  continually  reviewed  and  are  periodically  revised  and  reissued.  ISO  9000 
is  being  expanded  beyond  the  original  manufacmring  focus  to  service  industries  and  software  de¬ 
velopment  and  is  expected  to  be  aligned  more  closely  with  the  precepts  of  continuous  quality  im¬ 
provement  and  total  quality  management. 
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♦  consistent  perfonnance; 

♦  provision  of  a  framework  for  measuring  performance  and  improving  proc¬ 
esses; 

♦  continuous  improvement  in  efficiency  and  effectiveness; 

♦  enhancement  of  the  ability  to  deal  with  a  changing  business  environment; 
and 

♦  establishment  of  a  unified,  purposeful  system  composed  of  interrelated 
business  processes. 

A  survey  conducted  in  1996  by  the  Construction  Industry  Research  and  Informa¬ 
tion  Association  confirmed  those  benefits.  Surveyed  companies  also  reported  that 
having  an  effective  quality  management  system  has  enabled  them  to 

♦  clarify  responsibilities, 

♦  reduce  errors, 

♦  improve  the  flow  of  information, 

♦  identify  management  issues  early, 

♦  reduce  the  time  needed  for  routine  matters,  and 

♦  improve  production.^ 

Most  benefits  of  operating  under  the  ISO  quality  system  are  largely  qualitative  and 
generally  are  not  easily  quantified  unless  appropriate  measures  are  carefully 
planned  in  advance.  However,  one  quantitative  measure  is  the  dramatic  growth  in 
the  number  of  U.S.  companies  that  have  become  registered  as  complying  with  ISO 
9000,  which  has  leaped  from  241  in  1992  to  nearly  21,000  in  1998,  as  shown  in 
Figure  1-1.  This  explosive  rate  of  growth  is  expected  to  continue  as  more  organi¬ 
zations  learn  about  the  benefits  of  ISO  9000. 

The  companies  attaining  ISO  9000  registration  are,  to  a  large  extent,  manufactur¬ 
ing  firms,  but  the  standard  is  equally  applicable  to  other  types  of  businesses  in  the 
private  sector  (such  as  professional  services  firms),  as  well  as  to  government  or¬ 
ganizations.  For  example,  a  large  number  of  architectural,  engineering,  and  con¬ 
struction  firms — ^many  of  which  are  Corps  partners — ^have  become  registered. 
Among  them  are  Bechtel  Group,  Inc.;  Brown  and  Root,  Inc.;  Huor  Daniel;  Jacobs 
Engineering  Group,  Inc.;  Raytheon  Engineers  and  Constructors;  and  Stone  and 


^  Construction  Industry  Research  and  Information  Association,  Quality  Management  in  Con¬ 
struction-Survey  of  Experiences  with  BS  5750:  Report  of  Key  Findings,  Special  Publication  132, 
London:  CIRIA,  1996. 
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Webster  Engineering  Corporation.  These  companies  are  setting  the  pace  for  the 
industry. 


Figure  1-1.  Number  of  U.S.  Organizations  with  ISO  9000  Certification 


1992  1993  1994  1995  1996  1997  1998 

Year  (2nd  Qtr) 


Source:  “ISO  9000  Registrations  Continue  North  American  Climb,”  Quality  Systems  Update, 
July  1998,  p.  1. 

With  several  major  engineering  and  construction  firms  already  registered  (and 
many  more  actively  implementing  ISO-compliant  quality  systems),  industry  ex¬ 
perts  believe  that  ISO  9000  registration  may  become  a  major  factor  in  the  selec¬ 
tion  of  engineering  and  construction  firms  by  customers  in  both  the  private  and 
public  sectors  in  the  not-to-distant  future.  One  example  of  a  large  project  for 
which  the  contract  specified  that  firms  be  registered  as  having  an  ISO  quality  sys¬ 
tem  is  the  $1.3  billion  reconstruction  of  the  1-15  corridor  for  the  2002  Olympics  in 
Salt  Lake  City. 

USAGE  AND  ISO  9000 

Early  work  done  by  the  Corps  on  ISO  9000  focused  on  interpreting  how  the  ISO 
quality  management  system  would  apply  to  its  engineering  and  construction  ac¬ 
tivities  and  on  developing  a  strategy  for  integrating  its  existing  quality  initiatives 
into  the  ISO  quality  system.'^  Recognizing  the  many  potential  advantages  of 
adopting  the  ISO  quality  system,  USACE  headquarters  selected  four  engineering 
and  two  construction  field  sites  to  participate  in  a  pilot  program  to  test  the  concept 
of  quality  system  registration.  USACE  asked  LMI  to  assist  the  field  sites  with  the 
pilot  program.  LMI  supplemented  its  staff  with  the  Victoria  Group,  a  firm  that 


4 

Logistics  Management  Institute,  Toward  a  World-Class  Engineering  Organization — Making 
ISO  9000  the  Foundation  to  Quality  Management,  Report  CE308R1,  Jeffrey  A.  Hawkins  and 
James  L.  Hathaway,  April  1994;  Road  to  Engineering  Excellence:  ISO  9000  Blueprint  to  Success, 
Report  CE308RD1,  Jeffrey  A.  Hawkins,  April  1995;  Construction  Management  Excellence:  Using 
ISO  9000  to  Improve  Quality  Systems,  Report  CE501R1,  Jeffrey  A.  Hawkins,  November  1995. 
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specializes  in  ISO  9000  consulting  services.  This  report  assesses  the  Corps’  im¬ 
plementation  of  ISO  9000  at  the  six  sites  and  the  costs  and  benefits  of  doing  so. 

Report  Organization 

In  Chapter  2  of  the  report,  we  briefly  describe  the  ISO  quality  system  and  the  typi¬ 
cal  ISO  9000  certification  process.  Chapter  3  discusses  the  approach  taken  by  the 
Corps  sites  to  become  certified,  as  well  as  die  approach  we  used  to  identify  the 
costs  and  benefits  of  ISO  9000  certification.  Our  assessment  of  the  pilot  program, 
along  with  discussions  of  lessons  learned,  is  contained  in  Chapter  4,  and  in  Chap¬ 
ter  5,  we  present  our  conclusions  about  the  applicability  of  the  ISO  quality  system 
to  the  Coips  and  recommend  some  steps  that  US  ACE  should  take  if  it  decides  to 
implement  ISO  9000  throughout  the  Corps. 
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Chapter  2 

ISO  9000 


Many  people  in  the  Corps  have  heard  of  the  ISO  9000  quality  management  sys¬ 
tem.  But  exactly  what  the  system  is,  how  it  serves  as  a  means  to  improve  customer 
satisfaction,  what  the  Corps  must  do  to  become  registered  as  an  ISO-compliant 
organization,  and  how  an  ISO  quality  system  leads  to  improvement  of  USACE 
business  processes  are  not  well  understood.  In  this  chapter,  we  briefly  discuss 
those  concepts. 

ISO  Quality  System 

ISO  9000  comprises  three  different  models,  or  standards,  for  quality  manage¬ 
ment — ^ISO  9001,  ISO  9002,  and  ISO  9003.^  The  standard  with  which  an  organi¬ 
zation  should  comply  depends  on  its  functions.  ISO  9001,  the  most  compre¬ 
hensive  and  rigorous  of  the  ISO  quality  standards,  is  the  model  to  be  used  by  or¬ 
ganizations  involved  in  all  aspects  of  producing  a  product,  from  design  and  devel¬ 
opment  through  production  to  installation  and  servicing;  this  model  applies  to 
USACE  engineering  activities.  {Product  includes  services,  processed  materials, 
hardware,  software,  or  a  combination  thereof,  and  may  be  tangible,  such  as  proc¬ 
essed  materials,  or  intangible,  such  as  knowledge  or  concepts.)  ISO  9002  is  the 
model  to  be  used  by  organization’s  involved  in  production  with  no  design  func¬ 
tion,  such  as  USACE  construction  activities.  ISO  9003  applies  to  inspection  and 
testing  organizations. 

The  ISO  quality  standards  are  based  on  a  set  of  common  functional  elements  that 
contribute  to  an  organization’s  ability  to  meet  its  customer’s  expectations  and  to 
prevent  deviations.  Those  elements,  and  a  brief  interpretation  of  them,  are  as  fol¬ 
lows: 

1 .  Management  Responsibility.  The  organization’ s  management  must  be  fuUy 
committed  to  establishing  and  maintaining  a  quality  management  system.  It 
must  have  a  quality  policy  and  measurable  business  objectives,  it  must  assign 
properly  trained  personnel,  it  must  provide  the  resources  necessary  to  imple¬ 
ment  its  policy,  and,  periodically,  it  must  review  the  progress  being  made  to¬ 
ward  achieving  its  stated  quality  policy  and  business  objectives. 


'  The  ISO  9000  standards  have  their  origins  in  both  the  United  States  and  Europe.  They  share 
a  common  background  with  such  standards  as  MIL-Q-9858,  NATO  AQAPl,  and  the  British  Stan¬ 
dards  Institutes  series  of  quality  standards  (BS  5750). 
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2.  Quality  System.  The  organization  must  develop,  document,  and  implement 
policy,  procedures,  and  appropriate  work  instractions  covering  each  applicable 
element  of  the  ISO  quality  management  system. 

3.  Contract  Review.  Before  it  agrees  to  any  contractual  arrangement  with  a  cus¬ 
tomer,  the  organization  must  work  closely  with  the  customer  to  ensure  that  the 
customer’s  requirements  are  clearly  defined,  the  roles  of  both  parties  are  fully 
understood,  the  resources  are  available  to  fulfill  the  contract,  and  the  organi¬ 
zation  can  deliver  the  requested  product  or  service. 

4.  Design  Control.  At  key  stages  during  tfie  design  process  (project  conception 
through  design  completion),  the  organization  must  formally  review  the  design 
to  ensure  that  it  will  result  in  a  product  that  meets  the  customer’s  require¬ 
ments.  The  review  must  be  done  by  appropriate  technical  professionals  and 
customer  representatives. 

5.  Document  and  Data  Control.  The  organization  must  ensure  that  staff  members 
use  only  current  documents  (for  example,  plans  showing  the  latest  design 
changes)  and  data  (current  building  codes)  related  to  producing  a  quality 
product  or  service.  That  is,  it  must  ensure  that  staffers  never  use  invalid  or  ob¬ 
solete  documents.  Applicable  federal  statutes,  USAGE  engineering  regulations 
and  guide  specifications,  and  project  specifications,  as  well  as  quality  system 
procedures,  are  examples  of  sueh  documents.  The  organization  also  must  en¬ 
sure  that  all  documents  and  data  are  readily  available. 

6.  Purchasing.  Whenever  it  purchases  goods  and  services  needed  to  fulfill  the 
contract  with  its  customer,  the  organization  must  clearly  define  what  is  being 
purchased  (for  example,  purchase  orders  must  contain  adequate  technical 
data),  ensure  that  the  supplier  is  capable  of  delivering  that  needed  product  or 
service,  and  verify  that  what  the  supplier  delivers  conforms  to  what  was  or¬ 
dered. 

7 .  Control  of  Customer-Supplied  Product.  The  organization  must  ensure  that  it 
does  not  lose  or  damage  any  items  supplied  by  the  customer.  Items  reeeived 
must  be  suitable  for  their  intended  use. 

8.  Product  Identification  and  Traceability.  The  organization  must  be  able  to 
readily  identify  its  produets  for  a  particular  customer  during  any  stage  of  pro¬ 
duction,  delivery,  or  installation.  An  effeetive  identification  system  should  en¬ 
able  the  organization  to  track  the  development  history  of  any  product  or 
service. 

9.  Process  Control.  The  organization  must  identify  all  business  practices  that  can 
affect  quality,  and  it  must  establish  appropriate  controls  as  well  as  provide  the 
tools  necessary  to  carry  them  out.  These  practices  range  from  the  availability 
of  detailed  procedures  and  work  instructions  and  of  appropriate  equipment 
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(e.g.,  CADD  systems)  to  the  assignment  of  personnel  to  those  projects  that 
best  take  advantage  of  their  knowledge,  skill,  and  training. 

10.  Inspection  and  Testing.  To  ensure  that  the  completed  product  or  service  will 
be  acceptable  to  the  customer,  the  organization  must  set  acceptance  criteria, 
review  all  work — whether  done  in-house  or  by  subcontractors — against  those 
criteria,  and  specify  the  steps  to  be  taken  if  the  criteria  are  not  met. 

11.  Control  of  Inspection,  Measuring,  and  Test  Equipment.  The  organization  must 
use  equipment  that  produces  accurate  results  consistently;  the  equipment  must 
meet  internationally  or  nationally  recognized  standards  for  calibration. 

12.  Inspection  and  Test  Status.  The  organization  must  have  a  system  for  identify¬ 
ing  the  pass/fail  status  of  each  product  that  it  inspects  or  tests. 

13.  Control  of  Nonconforming  Product.  The  organization  must  identify  products 
that  do  not  comply  with  the  customer’s  requirements  and  must  have  proce¬ 
dures  for  handling  those  products. 

14.  Corrective  and  Preventive  Action.  The  organization  must  take  steps  to  prevent 
problems  from  recurring. 

15.  Handling,  Storage,  Packaging,  Preservation,  and  Delivery.  The  organization 
must  protect  the  product  from  being  damaged  or  deteriorating  before  its  deliv¬ 
ery  or  use. 

16.  Control  of  Quality  Records.  The  organization  must  have  a  documented  system 
for  effectively  managing  all  quality  records  required  by  the  ISO  standard  and 
its  procedures. 

17.  Internal  Quality  Audits.  Periodically,  the  organization  must  review  its  entire 
quality  system  to  ensure  that  procedures  are  being  consistently  followed  as 
documented. 

18.  Training.  The  organization  must  develop  a  plan  for  each  employee  that  speci¬ 
fies  what  knowledge  and  training  is  appropriate  for  his  or  her  job,  and  it  must 
keep  records  of  what  training  each  employee  has  taken. 

19.  Servicing.  The  organization  must  service  the  product  it  delivers  to  the  cus¬ 
tomer  (if  servicing  is  specified  in  the  contract)  and  verify  that  it  meets  speci¬ 
fied  requirements. 

20.  Statistical  Techniques.  The  organization  must  identify  any  processes  that 
could  benefit  from  the  use  of  statistical  techniques  and,  where  appropriate, 
make  sure  the  techniques  employed  are  properly  used. 
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Appendix  A  describes  the  elements  in  more  detail,  focusing  on  how  they  apply  to 
USAGE  engineering  and  construction  activities.  A  careful  reading  of  the  elements 
demonstrates  that  the  ISO  quality  management  system  “is  not  simply  an  inspec¬ 
tion  process  to  eliminate  any  parts  or  services  that  do  not  meet  a  specific  set  of 
requirements.  Under  ISO  9000,  quality  is  ‘built  in,’  not  ‘inspected  in.’”^  Quality  is 
not  something  you  do,  it  is  the  way  you  do  things  and  is  the  result  of  controlled 
processes. 


The  specific  elements  that  an  organization  must  address  to  become  ISO  certified 
vary  from  model  to  model.  Organizations  wishing  to  become  certified  as  comply¬ 
ing  widi  ISO  9001  must  address  the  requirements  of  all  20  elements,  while  those 
seeking  ISO  9002  certification  must  address  the  requirements  of  19  elements — aU 
except  design  control.  Otherwise,  the  ISO  9001  and  ISO  9002  requirements  are 
identical.  In  contrast,  ISO  9003  is  considerably  less  rigorous;  organizations  must 
address  just  16  of  the  20  elements,  and  the  requirements  for  meeting  most  of  the 
remaining  elements  are  less  stringent  than  those  for  ISO  9001  and  9002.  (Because 
ISO  9003  does  not  apply  to  Corps  activities,  we  do  not  discuss  it  further.) 

The  elements  of  the  ISO  quality  management  system  should  be  considered  as  a 
whole  and  in  the  context  of  their  functions  and  intent.  Generally,  the  functions 
affect  one  of  five  different  business  components — customers,  management  and 
administration,  suppliers  (A-Es,  contractors,  vendors),  process  control,  and  quality 
assurance.  This  concept  is  illustrated  in  Figure  2-1. 

Figure  2-1.  Relationship  of  ISO  9000  Elements  to  Business  Components 
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Source:  ABS  Integrated  Services,  Inc.,  Houston,  TX. 

Note:  Some  elements  have  functions  that  relate  to  more  than  one  area.  For  example,  man¬ 
agement  responsibility  applies  to  the  customer  (management  review)  as  well  as  to  administra¬ 
tion/management  (resources  and  system  review). 


^  Donald  A.  Sanders  and  C.  Frank  Scott,  How  to  Pass  Your  ISO  Audit,  American  Management 
Association,  1994,  p.  4. 


2-4 


ISO  9000 


In  addition  to  the  standards  describing  the  requirements  of  the  three  quality  system 
models,  ISO  9000  encompasses  a  number  of  other  documents  that  contain 
additional  information  to  help  an  organization  establish  and  maintain  its  ISO- 
compliant  quality  system.  The  ISO  9000  family  of  models  and  guidance  is 
depicted  in  Figure  2-2. 


Figure  2-2.  ISO  9000  Family 
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Quality  Systems — Model  for  Quality  Assurance  in  Final  Inspection  and  Test 
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tion,  use,  and  application  of  ISO  9001,  ISO  9002,  and  ISO  9003) 
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Guidelines  for  Developing  Quality  Manuals 
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Compliance  with  the  standard  ensures  that  the  organization’s  management  system 
addresses  all  of  the  potential  sources  of  customer  dissatisfaction.  When  all  em¬ 
ployees,  including  management,  understand  and  fulfill  their  roles,  the  system  be¬ 
comes  effective  and  serves  the  good  of  all.  Using  the  system  leads  to  reduced  cost 
through  systematic  reduction  of  variation. 

Registration  of  Quality  System 

To  become  registered  as  having  an  ISO-compliant  quality  system,  an  organization 
must  have  processes  in  place  that  enable  it  to  address  the  requirements  of  each 
applicable  element.  In  addition,  it  must  document  those  processes,  and  it  must 
demonstrate  that  it  follows  them.  In  short,  it  must  say  what  it  does  and  do  what  it 
says.  And  the  adequacy  of  its  quality  system  must  be  assessed  and  periodically 
audited  by  an  independent,  accredited  third  party,  known  as  a  quality  system  reg¬ 
istrar.  When  an  organization’s  system  conforms  to  the  registrar’s  interpretation  of 
the  ISO  9000  standard,  the  registrar  issues  a  “certificate  of  registration.”  The  steps 
to  establish  an  ISO  quality  system  are  described  in  the  following  subsections. 

Preparation  of  an  ISO  9000  Project  Management  Plan 

Once  an  organization  decides  to  implement  an  ISO  quality  system,  management 
should  prepare  a  plan  describing  how  the  organization  is  going  to  get  the  job  done. 
In  addition,  it  should  appoint  and  train  a  management  representative  and  a  man¬ 
agement  team. 

Identification  of  Quality  System  Processes 

The  next  step  is  to  analyze  and  define  all  of  its  existing  processes  and,  where  nec¬ 
essary,  develop  new  processes.  Organizations  preparing  for  registration  to  the 
standard  frequently  wrestle  with  the  question  of  what  a  process  is.  In  ISO  9000,  a 
process  is  a  collection  of  activities  that  takes  one  or  more  inputs  and  creates  an 
output  that  is  of  value  to  a  customer.  The  set  of  activities  required  to  build  a 
house,  the  set  of  activities  required  to  train  a  staff,  and  the  set  of  activities  re¬ 
quired  to  conform  to  a  particular  ISO  9000  element  are  all  processes. 

Most  organizations  have  business  processes  that  address,  at  least  to  some  extent, 
many  of  the  ISO  elements.  However,  those  processes  have  evolved  over  several 
years,  often  in  an  effort  to  solve  problems  as  they  arise  but  also  in  response  to 
management  whim,  legislative  requirements,  or  other  drivers.  As  a  result,  most 
organizations  have  processes  with  activities  that  do  not  add  value  and  may  in  fact 
hinder  efficiency  or  activities  that  are  duplicated  in  more  than  one  process.  Devel¬ 
oping  an  ISO  quality  system  requires  that  the  organization  make  a  concerted  effort 
to  define  each  of  its  processes;  it  does  not  require  business  process  reengineering. 
Typically,  however,  when  an  organization  makes  a  concerted  effort  to  identify  and 
define  all  of  its  processes,  it  finds  activities  that  do  not  add  value,  activities  in  one 
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process  that  duplicate  activities  in  another  process,  and  other  activities  that  can 
easily  be  adjusted  to  improve  its  processes. 

Documentation  of  Processes 

After  it  has  identified  all  of  its  processes,  the  organization  must  document  them. 
Specifically,  it  must  prepare  documents  describing  what  processes  and  procedures 
it  uses  to  dehver  the  product  or  service,  how  quality  is  ensured,  and  how  customer 
requirements  are  consistently  met.  Adequate  documentation  demonstrates  to  the 
third-party  auditor  that  the  quality  system  has  been  planned  and  that  each  element 
of  the  standard  is  sufficiently  addressed.  The  documentation  does  not  have  to  be 
onerous  and  should  be  appropriate  for  the  situation.  Too  much  documentation  is 
more  troublesome  than  not  enough. 

Figure  2-3  illustrates  the  four-level  documentation  structure  suggested  by  the  ISO 
9000  standards. 


Figure  2-3.  ISO  9000  Documentation  Hierarchy 
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At  the  top  of  the  documentation  pyramid  is  quality  policy  as  documented  in  a 
quality  manual.  The  quality  manual  describes  the  organization’s  policy  with  re¬ 
spect  to  each  applicable  element  and  typically  is  20  to  30  pages  long.  Level  2 
documentation  addresses  the  quality  system  procedures  and  should  relate  to  the 
policy  requirements  as  stated  in  the  quality  manual.  While  most  of  USAGE’ s 
quality  procedures  are  broadly  described  in  directives  and  regulations,  the  ISO 
auditors  will  be  concerned  with  how  those  regulations  and  statutes  are  imple¬ 
mented  at  the  local  level.  The  description  of  quality  procedures  does  not  need  to 
be  lengthy;  it  is  enough  to  describe  simply  what  is  to  be  accomplished.  Level  3, 
work  instructions,  deals  with  how  each  required  procedure  is  to  be  accomplished. 
Those  instructions  form  the  bulk  of  the  documentation.  Not  every  procedure  needs 
associated  work  instructions,  but  work  instructions  should  be  documented  for  any 
procedures  directly  affecting  the  quality  of  the  work  effort.  Level  4  documentation 
comprises  the  records,  or  proof,  that  the  organization  has  implemented  practices 
in  accordance  with  the  stated  quality  policy  and  procedures. 

An  appropriate  documentation  system  is  the  foundation  for  a  successful  quality 
management  system  and  provides  training  in  practiced  procedures  for  new  per¬ 
sonnel  to  ensure  consistency  of  performance. 

Implementation  of  Quality  System 

The  next  step  is  to  implement  the  quality  system  as  documented.  The  organization 
must  train  trainers,  and  it  must  train  employees  to  ensure  that  they  understand  the 
quality  system  and  how  to  operate  under  it. 

The  organization  must  then  prove  (usually  verified  through  observation  and  qual¬ 
ity  records)  tiiat  the  personnel  or  systems  responsible  for  executing  its  docu¬ 
mented  policies  and  procedures  are  doing  so: 

♦  The  organization  must  have  evidence  that  quality  records  have  been  estab¬ 
lished  and  that  they  are  legitimate. 

♦  The  organization  must  evaluate  the  effectiveness  of  its  processes  with  re¬ 
spect  to  its  documented  procedures  and  customer  requirements.  Typically, 
an  organization  selects  and  trains  a  group  of  employees  to  conduct  peri¬ 
odic  audits.  The  purpose  of  the  audits  is  to  provide  opportunities  for  im¬ 
provement  and  to  review  the  quality  system  and  to  identify  areas  in  which 
procedures  are  not  being  followed,  not  to  point  fingers  but  rather  to  enable 
the  organization  to  learn  what  does  not  work  as  well  as  it  could.  The  or¬ 
ganization  can  then  determine  why  the  procedures  are  not  being  followed. 
For  example,  in  some  cases,  employees  may  not  be  following  a  certain 
procedure  because  they  think  it  is  unimportant;  in  other  cases,  they  may 
not  understand  what  is  required.  Or  the  procedure  itself  may  be  inefficient 
or  not  accurately  documented. 
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♦  When  it  identifies  a  problem,  the  organization  must  take  appropriate  ac¬ 
tion.  That  is,  it  must  have  a  system  for  addressing  and  resolving  current 
problems  and  preventing  potential  systemic  problems.  When  warranted, 
that  action  should  include  changing  existing  procedures  and  updating  rele¬ 
vant  documentation.  The  process  of  identifying  and  correcting  system  de¬ 
ficiencies  serves  as  a  springboard  for  continual  process  improyement. 

♦  Internal  Auditors  may  also  make  a  recommendation  to  improve  a  process 
even  when  they  find  that  it  is  being  followed  as  planned.  Since  auditors  are 
independent  of  the  processes  they  review,  they  may  identify  opportunities 
for  improvement.  Process  revisions  may  not  always  be  the  result  of  an 
audit  nonconformance. 

When  it  operates  with  an  ISO  9000-compliant  system  in  place  is  when  an  organi¬ 
zation  begins  to  see  the  benefit  of  all  of  its  work  in  documenting  its  processes. 
This  phase  can  vary  in  length  but  needs  to  be  long  enough  to  document  the  results 
of  operations,  internal  audits,  and  management  review  cycles  (normally  at  least  6 
months). 

After  the  quality  system  has  been  in  place  and  operating  long  enough  to  generate 
reports  of  internal  audits,  corrective  and  preventive  action  reports,  and  other  evi¬ 
dence  that  it  has  been  following  the  system,  the  organization  should  review  the 
system.  Typically,  the  system  review  is  best  done  by  the  trainers.  The  system  re¬ 
view’ s  purpose  is  to  make  sure  that  the  organization  is  ready  to  take  the  next 
step — apply  for  a  certificate  of  registration. 

Third-Party  Assessment 

When  management  is  confident  the  quality  system  is  working,  it  selects  an  ac¬ 
credited  registrar  to  assess  its  system.^  The  third-party  assessment  is  intended  to 
verify  that  the  organization  has  documented  and  implemented  a  quality  system 
that  meets  the  requirements  of  the  applicable  standard,  that  the  organization  has 
objective  evidence  of  its  compliance  with  its  documented  system,  and  that  the 
system  is  self-correcting  and  continuously  improving.  The  third-party  assessment 
has  the  following  components:  document  review,  preassessment  audit,  certifica¬ 
tion  audit  and  report,  corrective  actions,  and,  ultimately,  registration  as  being  cer¬ 
tified  in  compliance  with  the  standard. 

The  registrar’s  services  begin  with  a  review  of  the  quality  system  manual.  This 
review  is  conducted  from  the  registrar’s  office  before  an  on-site  preassessment 
audit.  The  combination  of  document  review  and  preassessment  audit  is,  in 
essence,  a  “feasibility”  study  to  determine  if  the  organization  is  ready  for  the 

^  The  role  of  the  registrar  is  so  important  that  we  recommend  that  one  company  serve  as  the 
registrar  for  all  districts  so  that  it  will  be  intimately  familiar  with  USAGE  functions,  constraints, 
and  terminology,  and  so  that  guidance  about  an  ISO  quality  system  will  be  consistent  across  the 
Corps. 
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full-blown  certification  audit.  The  information  generated  helps  identify  areas  that, 
at  a  glance,  may  be  weak  or  problematic  and  gives  the  organization  the 
opportunity  to  make  appropriate  adjustments  before  the  registrar  does  the  formal 
audit. 

The  formal  certification  audit  is  zm  independent,  systematic,  and  documented  ac¬ 
tivity  to  evaluate,  verify,  and  report  on  all  aspects  of  the  organization’s  compli¬ 
ance  with  the  requirements  of  the  standard.  The  audit  includes  observation  of 
daily  activities,  interviews,  and  reviews  of  related  records.  After  completing  the 
audit,  the  registrar  will  write  a  report  that  lists  the  objective  evidence  of  compli¬ 
ance  with  the  requirements  of  the  standard.  The  report  also  identifies  areas  of 
nonconformance  and  categorizes  them  as  either  minor  or  major.  If  too  many  areas 
of  nonconformance  exist,  the  auditors  will  recommend  that  the  organization  not 
be  certified  as  ISO  compliant.  Otherwise,  they  may  recommend  that  the  organiza¬ 
tion  be  approved  or  that  it  receive  conditional  approval  contingent  on  its  correct¬ 
ing  the  areas  of  nonconformance.  In  the  latter  case,  satisfying  the  auditors  that 
corrective  action  has  been  taken  in  areas  defined  as  nonconforming  generally  is 
aceompHshed  without  an  additional  on-site  visit  to  the  organization.  The  report  is 
reviewed  for  conformance  with  the  registrar’s  requirements  and  then  the  registrar 
issues  a  certificate  of  registration. 

After  receiving  its  certificate  of  registration,  the  organization  must  demonstrate, 
through  periodic  surveillance  audits  by  the  registrar,  that  it  is  continuing  to  use  its 
quality  system. 

ISO  9000  AS  A  Vehicle  for  Continuous 
Improvement 

When  an  organization  looks  at  how  it  does  business  in  the  context  of  the  elements 
of  the  ISO  quality  system,  it  sees  itself  in  a  whole  new  light.  In  fact,  the  power  of 
the  standard  is  to  a  large  extent  the  management  discipline  it  requires  and  nur¬ 
tures.  In  the  hands  of  solid  managers,  the  standard  promotes  an  understanding  of 
what  is  done  and  how  it  is  accomplished,  and  it  provides  a  structure  for  system 
reviews  and  corrective  and  preventive  actions.  That  structure  is  the  foundation 
necessary  for  continually  improving. 

Compliance  with  ISO  9000  ensures  that  the  groundwork  for  good  business 
practices  is  in  place  and  forces  organizational  understanding  and  adherence  to 
documented  practiees.  By  documenting  its  major  processes,  an  organization  is 
forced  to  consider  and  reach  agreement  on  what  those  processes  are,  to  identify 
the  value-added  steps,  and  to  eliminate  those  that  do  not  add  value.  It 
complements  an  organization’s  existing  quality  programs  and  eneourages  the 
organization  to  incrementally  and  continually  improve  its  business  processes 
through  a  system  of  identifying  problems  and  taking  corrective  and  preventive 
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actions.  As  the  implemented  system  matures,  the  documented  processes  are 
examined  and  evaluated  to  identify  more  effective  ways  of  doing  business. 

In  short,  ISO  9000  is  a  dynamic  and  comprehensive  system  that  can  be  used  by 
any  organization  to  ensure  the  quality  of  its  products  and  thus  the  satisfaction  of 
its  customer.  Quality  is  not  something  you  do;  it  is  the  way  you  do  things  and  re¬ 
sults  from  controlled  processes.  ISO  9000  is  about  managing  the  business  in  such 
a  way  that  daily  operations  are  continually  improved,  thus  making  the  organiza¬ 
tion  more  competitive.  The  appealing  aspect  of  the  standard  is  that,  by  focusing  on 
the  20  elementaiy  business  requirements,  engineering  and  construction  organiza¬ 
tions  can  establish  practical,  common  sense  quality  management  systems  to  en¬ 
sure  that  each  customer’s  requirements  are  met  and  that  die  organization’s 
fundamental  business  processes  are  continually  evaluated  and  improved. 
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Chapter  3 

Pilot  Program  Approach 


USAGE  began  the  ISO  9000  pilot  program  in  the  spring  of  1995.  Six  divisions 
participated;  Louisville  District’s  Engineering  Division  and  Construction  Divi¬ 
sion,  Savannah  District’s  Engineering  Division  and  Construction  Division,  Kan¬ 
sas  City  District’s  Engineering  Division,  and  Portland  District’s  Planning  and 
Engineering  Division.  This  chapter  discusses  the  approach  that  the  six  Corps  divi¬ 
sions  used  to  become  registered,  as  welt  as  the  approach  that  we  used  to  identify 
the  costs  and  benefits  of  ISO  9000  certification  at  those  sites. 

Certification  Approach 

Strategy 


The  basic  approach  used  by  USAGE  in  its  ISO  9000  pilot  program  was  the  same 
methodology  its  consultants  had  used  with  private-sector  corporate  clients  such  as 
Ford  Motor  Company.  The  certification  strategy  included  three  phases:  planning, 
implementation,  and  registration.  Figure  3-1  portrays  the  strategy.  The  LMI  team 
facilitated  the  planning  and  implementation  phases;  the  registration  phase  is  the 
responsibility  of  the  registrar. 

Figure  3-1.  ISO  9000  Implementation  Strategy 
Planning  Phase 


Implementation  Phase 


Registration  Phase 
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Planning  Phase 


The  planning  phase  has  three  basic  modules:  a  2-day  introduction  and  implemen¬ 
tation  workshop,  a  3-day  gap  analysis  and  action  plan,  and  a  1-day  action  plan 
workshop.  The  1-day  awareness  workshop  is  optional.  The  recommended  elapsed 
time  for  this  phase  is  90  days  but,  during  die  pilot  program,  it  took  as  few  as  2 
months  and  as  long  as  10  months  to  accomplish. 

The  awareness  workshop  was  targeted  to  the  management  team.  It  was  intended  to 
be  available  only  for  those  divisions  that  felt  they  needed  to  generate  enthusiasm 
and  support  in  the  senior  management  ranks  for  implementing  a  system  to  im¬ 
prove  quality.  The  objectives  were  to  increase  awareness  of  quality  issues  and  dis¬ 
cuss  the  importance  and  relevance  of  the  ISO  9000  standard.  The  agenda  for  the 
workshop  included  an  introduction  to  quality  and  quality  management,  discus¬ 
sions  of  why  quality  should  be  managed  using  ISO  9000  and  why  quality  is  im¬ 
portant  to  USAGE  engineering  and  construction  activities,  an  introduction  to  ISO 
9000,  and  discussions  of  the  link  between  Total  Army  Quality  and  ISO  9000  and 
how,  together,  they  will  enable  USAGE  to  satisfy  its  customers. 

The  ISO  introduction  and  implementation  workshop  was  attended  by  the  ISO 
project  steering  team,  the  management  representative,  and  the  internal  audit  team 
(if  it  had  been  established).  The  objectives  were  to  provide  a  general  understand¬ 
ing  of  the  ISO  9000  standards  as  they  apply  to  USAGE,  develop  and  refine  the 
implementation  strategy  for  the  division  and  reach  agreement  on  the  dates  for 
completing  activities,  and  begin  addressing  ISO  9000  requirements  for  manage¬ 
ment  responsibility.  The  agenda  for  the  workshop  included  the  following  topics: 
interpretation  of  ISO  9000  in  USAGE,  implementation/registration  process,  man¬ 
agement  representative  and  steering  team  responsibilities,  and  awareness  strategy. 
In  addition,  during  the  workshop,  attendees  began  drafting  the  scope  of  registra¬ 
tion  and  the  quality  policy  and  objectives  for  their  division,  along  with  an  applica¬ 
bility  matrix  and  high-level  process  flow  charts. 

The  purpose  of  the  gap  analysis,  which  was  performed  by  the  LMI  team,  was  to 
identify  current  quality  system  shortfalls  and  actions  needed  to  bring  the  system 
into  compliance  with  ISO  9000  requirements.  LMI  reviewed  the  existing  quality 
systems  in  the  context  of  the  scope  of  registration  agreed  upon  during  the  intro¬ 
duction  and  implementation  workshop.  The  activity  included  an  initial  briefing, 
extensive  interviews,  and  an  exit  briefing  on  the  initial  findings.  The  gap  analysis 
typically  took  3  days. 

The  results  of  the  gap  analysis  were  documented  in  a  report  and  used  to  develop  a 
1-day  action  plan  workshop.  The  workshop  was  attended  by  the  steering  team, 
management  representative,  and  internal  auditors  (if  they  had  been  selected).  The 
objectives  of  the  workshop  were  to  finalize  planning  activities — ^by  presenting  gap 
results,  developing  a  clear  understanding  of  what  is  needed  to  achieve 
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certification,  and  estimating  the  labor  required — and  to  gain  commitment  to 
moving  forward  into  the  implementation  phase. 

Implementation  Phase 

The  implementation  phase  has  four  modules:  a  2-day  procedures  and  documenta¬ 
tion  workshop,  periodic  consulting  visits,  a  3-day  internal  auditor  training  class, 
and  a  comprehensive  system  review  just  before  beginning  the  registration  phase. 
The  total  elapsed  time  for  this  phase  was  expected  to  be  approximately  6  montiis, 
but  the  three  divisions  that  had  achieved  certification  when  this  report  was  written 
took  6  to  18  months. 

The  2-day  procedures  and  documentation  workshop  kicked  off  flie  implementa¬ 
tion  phase  of  the  program  for  the  pilot  sites.  The  workshop  was  attended  by  the 
steering  team,  management  representative,  and  quality  action  team  members.  De¬ 
pending  on  how  the  division  was  going  to  execute  the  procedures  review  and 
documentation  development,  this  workshop  could  be  attended  by  quite  large 
number  of  staff  members,  many  of  whom  had  not  attended  the  previous  work¬ 
shops  and  were  not  oriented  to  the  ISO  9000  standard.  The  objective  of  the  work¬ 
shop  was  to  provide  the  people  that  were  going  to  do  the  work  with  the  training 
needed  to  independently  develop  procedures  and  documentation  that  would  serve 
as  the  foundation  for  their  quality  system  manual. 

The  consulting  visits  as  conceived  in  the  strategy  were  to  occur  once  between  the 
procedures  and  documentation  workshop  and  the  internal  auditor  training,  then 
after  the  training  and  before  the  system  review.  The  purpose  of  the  consulting  vis¬ 
its  was  to  review  the  documentation  in  the  context  of  the  requirements  of  the 
standard.  However,  during  flie  visits,  the  consultants  provided  advice  on  a  variety 
of  issues  and,  in  fact,  made  several  additional  consulting  visits. 

The  internal  auditor  workshop  was  intended  to  train  the  internal  auditing  team  to 
conduct  effective  internal  audits  of  the  quality  system.  The  2-day  workshop  in¬ 
cluded  an  introduction  to  the  standard  and  its  interpretation;  an  explanation  of 
auditing  requirements,  objectives,  process,  and  intent;  and  discussions  about  col¬ 
lecting  objective  evidence,  planning  and  conducting  the  audit  and  writing  the  re¬ 
port,  and  conducting  practice  audits. 

Registration  Phase 

The  registration  phase  generally  includes  a  review  of  the  documentation,  a  pre¬ 
assessment  audit,  and  a  compliance  audit.  The  compliance  audit  is  followed  by 
periodic  surveillance  audits,  typically  at  6-month  intervals.  Although  most  regis¬ 
trars  prefer  the  6-month  cycle,  the  surveillance  audit  interval  is  subject  to  negotia¬ 
tion  and  can  stretch  as  long  as  12-month  intervals. 
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Elapsed  Times 


The  total  time  each  of  the  divisions  experienced  in  the  framework  of  the  strategy 
diagramed  in  Figure  3-1  is  displayed  in  Table  3-1.  As  the  table  shows,  the  times 
required  to  attain  registration  ranged  from  22  months  to  34  months  (assuming  Sa¬ 
vannah  Construction  attains  certification  as  planned).  One  site,  Kansas  City  Engi¬ 
neering,  has  been  operating  under  its  ISO  quality  system  since  spring  1998  but  has 
not  yet  set  a  schedule  for  the  registration  phase  because  of  recent  organizational 
and  personnel  changes.  Another,  Savannah  Engineering,  has  not  made  a  sustained 
management  commitment  to  pursue  ISO  9000  and  is  not  working  on  developing 
its  quality  system  at  this  time. 


Table  3-1.  Elapsed  Preparation  Times 


District/division 

Planning 

Implementation 

Registration 

Total  time 

Louisville 

Engineering 

3/95-10/95 

11/95-4/96 

March  1997 

25  months 

Construction 

1/96-3/97 

4/97-12/97 

August  1998 

32  months 

Savannah 

Engineering 

3/95-10/95 

11/95-NA 

Not  planned 

NA 

Construction 

2/96-3/96 

3/96-9/97 

Fall  1998® 

34  months® 

Kansas  City 
Engineering 

5/95-1/96 

3/96-11/96 

Pending*’ 

NA 

Portland 

Planning  and 

11/95-1/96 

3/96-1 1/96 

August  1997 

22  months 

Engineering 

Note:  NA  =  Not  applicable. 

®  Planned. 

Pending  =  reconciiing  impact  of  organizational  changes. 


Internal  Costs 

Headquarters  paid  for  the  consulting  services  for  the  pilot  program.  However,  the 
divisions  incurred  indirect  labor  costs  and  incidental  expenses  for  software  or  in¬ 
ternal  promotional  literature;  they  also  incurred  the  cost  of  the  registrar’s  services. 
Table  3-3  shows  the  expenses  reported  by  the  divisions.  The  registrar  fees  shown 
in  the  table  vary  because  of  variations  in  the  scope  of  services  provided.  Some 
registrar  contracts  covered  one  site,  while  others  covered  multiple  sites,  and  some 
covered  periodic  surveillance  audits  at  12-month  intervals,  while  others  were  at  6- 
month  intervals.  Finally,  some  contracts  covered  only  1  year’s  services  while  oth¬ 
ers  were  for  3  years. 
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Table  3-2.  Division  Expenses  for  Pilot  Program 


District/division 

Labor  and 
incidentals 

Registrar 

Louisville 

Engineering 

$190,000 

$25,000 

Construction 

$125,000 

$60,000 

Savannah 

Engineering 

NA 

NA 

Construction 

$350,000 

$50,000® 

Kansas  City 

Engineering 

$168,000 

$12,000® 

Portland 

Planning  and  Engineering 

$264,000 

$28,000 

Note:  NA  =  not  applicable.  Savannah  did  not  complete  prepa¬ 
ration  for  certification. 

®  Estimated. 


Program  Status 

As  of  Sept.  1, 1998,  three  of  six  pilot  sites  had  achieved  ISO  9000  certification: 
Louisville  Engineering  (March  1997),  Portland  Planning  and  Engineering  (August 
1997),  and  Louisville  Construction  (August  1998).  Savannah  Construction  and 
Kansas  City  Engineering  have  implemented  their  systems  and  have  been  success¬ 
fully  operating  under  them  since  October  1997;  both  are  nearing  the  start  of  the 
registration  process.  Savannah  Engineering  has  not  had  the  management  support 
to  place  a  priority  on  implementing  the  standard. 

Louisville  Engineering  has  been  through  one  surveillance  audit,  and  Portland 
Planning  and  Engineering  has  been  through  two.  Both  are  continuing  to  learn  and 
benefit  from  their  systems. 

Assessment  Approach 

Throughout  the  pilot  program,  LMI  studied  the  experience  of  each  of  the  partici¬ 
pating  divisions  as  well  as  experiences  of  other  government  agencies  and  the  pri¬ 
vate  sector  for  purposes  of  comparison.  We  participated  in  most  of  the  training 
sessions,  interviewed  the  divisions’  management  during  different  phases  of  the 
program,  and  conducted  two  important  surveys  to  obtain  information  that  would 
help  USACE  decide  about  the  future  of  ISO  9000  certification  in  the  Corps. 

We  sent  the  first  survey  to  all  individuals  in  the  Louisville  Engineering  and  Port¬ 
land  Planning  and  Engineering  divisions  shortly  after  diey  became  registered  as 
ISO  compliant.  Louisville  and  Portland  received  their  certificates  in  March  and 
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August  1997,  respectively.  The  survey  asked  managers,  engineers,  technicians, 
and  administrative  employees  about  the  effectiveness  of  the  training  they  had  re¬ 
ceived  and  the  benefits  to  their  organizations  of  obtaining  their  ISO  9000  certifi¬ 
cates.  The  survey  was  stractured  using  Likert  batteries  in  which  respondents  are 
able  to  rank  answers  on  a  scale  from  one  to  five.  An  open-ended  conunent  re¬ 
sponse  area  was  provided  at  the  end  of  the  survey  to  capture  individual  opinions 
on  aspects  of  the  certification  process  not  covered  elsewhere  in  the  survey.  We 
received  responses  from  218  people  in  these  two  engineering  divisions.  A  copy  of 
the  survey  questionnaire  and  an  analysis  of  the  results  are  in  Appendixes  B  and  C, 
respectively. 

For  the  second  survey,  we  sent  questionnaires  (in  February  1998)  to  all  six  divi¬ 
sions  involved  in  the  ISO  9000  pilot  program.  The  questionnaires  contained  24 
open-ended  questions  covering  the  entire  process  of  preparing  for  ISO  9000  reg¬ 
istration,  from  the  organization  phase  to  the  final  external  audit  for  registration  (if 
registration  had  been  achieved  by  tiie  survey  date).  Among  other  questions,  we 
asked  how  long  the  process  took,  how  much  it  cost,  what  problems  were  encoun¬ 
tered,  what  the  negative  aspects  were,  what  the  positive  benefits  were,  and  how 
the  management  representative  and  internal  auditors  were  selected.  We  also  asked 
general  questions  about  how  the  process  was  managed,  what  process  was  used  for 
preparing  and  controlling  documentation,  how  initial  and  foUow-on  training  was 
done,  and  what  changes  were  recommended  for  implementing  the  certification 
process.  Finally,  we  asked  for  suggestions  for  districts  considering  pursuing  ISO 
9000  registration.  The  questions  and  the  corresponding  answers  provided  by  each 
of  the  four  divisions  that  responded  are  in  Appendix  D. 
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Chapter  4 

Assessment  of  Pilot  Program 


Our  focus  when  assessing  the  ISO  9000  pilot  program  was  on  aspects  that  would 
help  USAGE  decide  whether  the  quality  system  should  be  implemented  through¬ 
out  the  Corps.  In  this  chapter,  we  present  our  general  findings,  then  discuss  spe¬ 
cific  lessons  learned  during  the  certification  process.  We  then  discuss  the  costs 
and  benefits  of  implementing  the  ISO  quality  system  at  the  six  Corps  sites. 

General  Findings  and  Observations 

From  the  results  of  our  surveys,  it  is  clear  that  ISO  9000  has  improved  organiza¬ 
tional  effectiveness.  Pilot  sites  achieving  certification  indicated  that  the  move  to 
an  ISO  9000  quality  management  system  has  had  a  demonstrably  beneficial  effect 
on  the  Corps  of  Engineers  and  would  recommend  ISO  9000  registration  to  other 
districts.  These  districts  cited  a  variety  of  benefits,  the  most  prominent  being  a 
divisionwide  clarification  of  procedures  and  the  genesis  of  an  understandable, 
well-defined  operating  system. 

Managers  believe  the  ISO  9000  standards  provide  both  a  foundation  for  an  effec¬ 
tive  quality  management  system  and  a  basis  for  improving  customer  service.  Staff 
communications  have  improved,  better  work  products  are  being  delivered,  and 
“fire  drills”  are  becoming  less  common.  ISO  9000  imposes  order  and  discipline 
on  the  system  so  that  once  a  problem  surfaces,  management  must  address  the  is¬ 
sue  and  develop  a  solution  that  may  include  a  change  in  procedures,  additional 
training,  or  both. 

The  Corps  goal  is  to  provide  quality  engineering  products  and  services  to  its  con¬ 
sumers,  and  ISO  9000  has  simply  allowed  all  involved  parties  to  become  aware  of 
how  that  goal  is  achieved.  Internally,  ISO  9000  registration  has  forced  organiza¬ 
tions  to  truly  understand  their  operating  system.  Procedures  are  well  documented 
and  available  for  reference,  duties  and  responsibilities  are  clearly  assigned,  and 
the  elimination  of  unnecessary  backtracking  allows  the  maximization  of  division 
resources. 

The  centralizing  tendencies  of  operating  under  an  ISO  9000  system  permit  em¬ 
ployees  to  more  efficiently  produce  the  quality  service  that  is  associated  with 
USAGE.  Moreover,  ISO  9000  meshes  well  with  USAGE’ s  quality  assurance 
process — Total  Quality  Management — and  its  historic  philosophy  of  continuous 
improvement.  ISO  9000  provides  a  foundation  upon  which  TQM  can  have  quanti¬ 
fiable  results  and  excellent  performance  can  be  achieved,  even  attaining  the  Mal¬ 
colm  Baldrige  performance  level. 
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ISO  9000  is  beneficial  for  the  Corps  customer  in  that  it  requires  a  defined  scope  of 
work  prior  to  the  initiation  of  activity.  The  immediate  result  is  fewer  discrepancies 
between  customer  expectations  and  execution  of  contracted  services  by  USAGE, 
translating  to  the  subsequent  long-term  benefit  of  customer  satisfaction.  While 
ISO  9000  registration  is  likely  to  extend  a  competitive  edge  to  USAGE  now,  it  is 
also  likely  that  registration  will  be  an  advantage  well  into  the  future. 

Lessons  Learned 

Within  the  context  that  the  program  has  been  successful  (three  divisions  have 
achieved  certification  and  two  more  will  do  so  shortly),  the  approach  to  ISO  9000 
certification  can  be  improved  significantly.  In  particular,  the  time  it  takes  for 
completing  such  an  initiative  can  be  dramatically  reduced  from  that  experienced 
by  the  pilot  sites,  with  a  corresponding  reduction  in  expense. 

The  approach  to  certification  used  in  the  pilot  program  was  defined  in  three 
phases:  planning,  implementation,  and  registration.  The  planning  process,  with  its 
multiple  workshops  and  gap  analysis,  and  with  breaks  of  30  or  more  days  between 
each  event,  was  ineffective  and  typically  took  from  6  to  14  months.  Only  Portland 
conducted  the  planning  phase  in  less  than  6  months.  Because  of  the  amount  of 
time  for  this  phase,  some  divisions  lost  momentum.  In  addition,  because  of  the 
disruptive  nature  of  the  various  training  events  and  the  inconsistent  mix  of  at¬ 
tendees,  many  participants  in  the  pilot  program  complained  that  they  had  diffi¬ 
culty  understanding  what  the  intent  and  requirements  of  the  standard  are  and  how 
they  applied  to  the  Gorps,  as  well  as  what  tasks  had  to  be  done  to  prepare  for  certi¬ 
fication.  The  use  of  training  materials  that  were  too  generic  and  not  focused  on  the 
Gorps  also  contributed  to  the  lack  of  understanding. 

The  implementation  phase  as  presented  was  somewhat  misnamed  because  it  in¬ 
cluded  analysis  of  the  business  processes  and  preparation  of  documentation,  but 
did  not  focus  on  the  important  feature  of  operating  under  the  quality  management 
system.  The  organization  must  be  able  to  prove  to  the  registrar  that  it  is  using  the 
system  successfully.  Without  such  proof,  it  is  premature  to  begin  working  with  the 
registrar  to  achieve  certification  to  the  standard. 

The  following  are  some  specific  findings  particularly  relevant  to  the  implementa¬ 
tion  of  ISO  9000  Gorps-wide: 

♦  Total  and  continuous  management  support  is  important  to  the  successful 
development  of  an  ISO  9000  management  system. 

♦  The  undertaking  to  prepare  an  ISO-compliant  management  system  should 
be  planned  and  managed  just  like  a  high-priority  project. 
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♦  Districts  can  successfully  take  different  approaches  to  reviewing  their  ex¬ 
isting  business  processes  and  developing  their  quality  system  documenta¬ 
tion. 

♦  The  managers  and  other  staff  members  that  form  the  team  doing  the  work 
should  be  identified  and  receive  orientation  and  training  early  in  the  proj¬ 
ect  planning  and  mobilization  phase.  Elementary  ISO  training  is  important 
to  staff  members  at  all  levels  of  the  work  force. 

♦  Auditors  should  be  selected  carefully  because  they  play  an  important  role 
in  the  initial  success  of  the  program. 

♦  The  total  time  and  cost  to  attain  ISO  9000  certification  could  be  reduced. 

♦  The  ISO  certification  process  does  not  require  changes  in  the  organiza¬ 
tional  structure. 

♦  The  ISO  consultant  should  have  knowledge  of  USAGE  engineering  and 
construction  activities. 

♦  ISO  9000  and  existing  USAGE  quality  requirements  are  compatible. 

♦  Simultaneous  implementation  of  an  ISO  quality  management  system  for 
engineering  and  construction  divisions  in  the  same  district  should  be 
tested. 

Each  of  these  Endings  is  discussed  in  more  detail  in  the  following  subsections. 

Management  Support 

Pilot  organizations  repeatedly  emphasized  that  senior  management  commitment  to 
the  ISO  quality  management  system  is  a  must.  This  commitment  must  be  commu¬ 
nicated  to  supervisors  and  employees  at  all  levels  of  the  organization,  especially 
through  the  allocation  of  the  necessary  resources.  Gombining  ISO  program  re¬ 
sponsibility  with  normal  management  responsibility  proved  effective  in  allocating 
resources  and  shifting  the  divisions’  culture  to  the  ISO  management  system.  Divi¬ 
sions  tiiat  did  not  ahgn  responsibility  this  way  had  more  difficulty  making  prog¬ 
ress. 

Louisville  Engineering  was  the  first  to  achieve  certification.  That  division’s  top 
management  demonstrated  its  support  for  ISO  9000  by  naming  the  division’s  as¬ 
sistant  to  the  division  chief  as  the  management  representative.  That  individual  was 
able  to  write  ISO  goals  into  the  annual  business  plan,  allocate  resources,  set  pri¬ 
orities,  and  integrate  ISO  concepts  into  the  division’s  management  practices  early 
in  the  process.  Successful  efforts  to  sustain  momentum  included  the  use  of  fre¬ 
quent  staff  e-mails,  discussion  of  issues  and  updates  in  documented  procedmes 
during  weekly  management  and  staff  meetings,  and  provision  of  refresher  courses 
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as  needed.  These  communication  procedures  appeared  effective  for  building  sup¬ 
port  and  reminding  staff  members  of  their  vested  interest  in  achieving  and  main¬ 
taining  ISO  certification. 

Level  of  Importance 

The  effort  it  takes  to  prepare  an  ISO-compliant  management  system  is  significant. 
The  greater  the  effort  put  into  the  process,  the  greater  the  benefit  to  the  organiza¬ 
tion  is.  The  best  results  occur  when  the  undertaking  is  treated  like  a  high-priority 
project  with  thoughtful  planning,  scheduling,  and  execution. 

Documentation 

In  general,  the  pilot  organizations  established  multiple  teams  to  prepare  the  docu¬ 
mentation  necessary  to  meet  ISO  registration  requirements.  Both  the  number  of 
teams  and  the  team  compositions  varied.  Team  leaders  were  usually  branch  or 
section  supervisors,  and  team  members  were  selected  on  the  basis  of  their  subject 
knowledge.  PDot  organizations  used  their  LANs  or  intranets  to  varying  degrees  in 
the  development,  control,  and  dissemination  of  ISO  documentation.  The  most  ad¬ 
vanced  division  used  totally  electronic  means  for  document  dissemination  and 
control. 

The  management  representative,  ISO  coordinator,  and  consultant  played  key  roles 
in  the  preparation,  review,  editing,  revision,  and  publication  of  the  work  instruc¬ 
tions  and  materials  in  the  quality  manual.  Process  review  and  documentation  are 
critical  steps  on  the  path  to  ISO  certification  and  should  not  be  cut  short  to  save 
time  or  money. 

As  noted  by  some  of  the  pilot  sites,  the  procedure  review/documentation  phase 
would  have  been  shortened  considerably  had  a  template  or  “model”  quality  man¬ 
ual  been  available  for  reference  use.  The  Portland  Planning  and  Engineering  Divi¬ 
sion’s  ability  to  use  the  manuals  developed  by  the  Louisville  District  and  U.S. 
Coast  Guard  as  templates  saved  an  estimated  $8,000  to  $10,000.  Since  this  phase 
was  the  most  time-consuming,  and  since  the  quality  manuals  from  the  pilot  sites 
are  now  available  for  reference,  this  phase  of  the  registration  process  can  likely  be 
expedited  for  new  sites  that  want  to  attain  ISO  9000  certification  and  thus  save 
money.  Districts  considering  ISO  9000  registration  should  anticipate  that  their 
start-up  costs  likely  will  be  lower  than  those  for  the  pilot  organizations. 

ISO  Training 

The  steering  committee  and  execution  team  that  will  evaluate  the  existing  work- 
flow  processes  and  prepare  the  quahty  system  manual  should  receive  their  initial 
ISO  9000  orientation  and  documentation  training  at  the  same  time  and  early  in  the 
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project  planning  process.  The  pilot  sites  had  different  training  for  different  groups 
at  different  times  and  experienced  an  inefficient  and  arguably  ineffective  process. 

Also,  elementary  ISO  training  was  important  to  staff  members  at  all  levels  of  the 
work  force  because  they  had  little  or  no  previous  knowledge  of  the  elements  of  the 
ISO  standards  and  their  purposes.  Normal  employee  resistance  to  change  means 
awareness  and  orientation  training  is  extremely  important.  Eventually,  supervisors 
and  management  representatives  provided  orientation  and  follow-on  training  to 
the  work  force  on  numerous  occasions.  The  LAN  and  intranet  have  been  used  to 
augment  and  reinforce  this  training.  ISO  9000  orientation  has  been  incorporated 
into  new  employee  indoctrination.  Additional  or  refresher  training  is  scheduled  as 
necessary. 

Selection  of  Internal  Auditors 

Some  pilot  organizations  took  advantage  of  the  existing  quality  assurance  organi¬ 
zations  in  the  district  when  selecting  their  internal  auditors.  Louisville  Construc¬ 
tion  used  its  branch  that  was  performing  quality  assurance  visits  to  field  offices 
for  its  internal  auditors.  LouisviUe  Engineering  uses  the  district’s  Internal  Review 
Office  as  internal  auditors  and  has  trained  its  staff  to  perform  the  function  as  well. 
Savannah  Construction  and  Portland  selected  auditors  from  sections  within  their 
respective  divisions.  Auditing  is  an  ongoing  (not  constant)  responsibility  and  re¬ 
quires  individuals  not  only  with  excellent  knowledge  of  the  ISO  9000  standard  but 
also  with  the  right  attitude  and  interpersonal  skills.  Auditors  should  be  selected 
very  carefully;  they  play  an  important  role  in  the  initial  success  of  the  program. 

While  internal  auditors  are  professionally  trained,  some  felt  this  training  was  too 
general  and  often  missed  the  mark.  Internal  auditors  gain  the  most  knowledge  and 
best  experience  by  actually  conducting  audits.  Auditors  selected  from  the  Internal 
Review  Office  to  be  ISO  internal  auditors  naturally  tend  to  adapt  more  easily  to 
ISO  audit  requirements. 

This  training  should  be  provided  after  the  ISO  9000-compliant  quality  manage¬ 
ment  system  is  implemented  because  the  trainees  should  begin  immediately  to 
audit  the  system.  In  one  instance,  providing  the  training  too  soon  required  that  the 
auditors  be  retrained  when  their  services  were  needed.  The  individuals  selected 
for  this  assignment  need  to  fit  specific  skill  requirements  and  be  able  to  perform 
audits  periodically  as  a  routine  part  of  their  normal  workload. 

The  auditing  function  is  important  to  the  very  purpose  of  the  standard.  TTie  entire 
system  must  be  audited  by  the  internal  auditors  prior  to  certification,  and  there 
must  be  documented  evidence  that  the  function  is  performing  properly.  Continu¬ 
ous  improvement  is  an  attribute  of  the  ISO  9000  standard  that  distinguishes  it 
from  all  other  quality  programs  and  is  indeed  fundamental  to  its  success. 
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Time  and  Cost  to  Attain  ISO  9000  Certification 

In  1995  when  the  pilot  program  was  initiated,  it  was  generally  thought  that  regis¬ 
tration  could  be  achieved  in  12  to  14  months.  However,  for  the  four  pilot  organi¬ 
zations  reporting,  the  time  to  acquire  ISO  certification  ranged  from  15  to  33 
months,  with  an  average  of  23  months  (see  Table  3-1).  The  dramatic  difference 
between  expectation  and  reality  can  be  explained  principally  by  the  fact  that  this 
had  not  been  done  before  with  the  Corps,  so  a  lot  of  new  ground  needed  to  be 
covered.  In  addition,  the  training  was  not  tailored  to  meet  USAGE  needs.  How¬ 
ever,  the  amount  of  preparation  time  also  contributed  substantially  to  the  total 
time  required.  Respondents  to  our  February  1998  questionnaire  indicated  that  they 
spent  from  6  to  14  months  at  the  beginning  trying  to  figure  out  what  to  do.  A  re¬ 
duction  to  30  days  for  developing  the  management  plan,  conducting  initial  train¬ 
ing,  and  getting  started  on  process  reviews  is  entirely  achievable,  saving  both  time 
and  money.  By  dramatically  reducing  the  schedule  for  initial  organization  work, 
USAGE  organizations  can  realistically  expect  to  achieve  registration  in  12  to  18 
months  if  they  do  not  experience  disruptions  because  of  urgent  or  emergency 
work.^  A  more  compressed  schedule  also  would  make  it  easier  to  keep  the  work 
force  highly  focused  on  organizing  and  implementing  the  ISO  9000  standards.  Fi¬ 
nally,  by  expediting  the  process,  costs  could  be  reduced.  Gosts  for  the  entire  reg¬ 
istration  process,  including  the  preparation  of  documentation,  training  of  the 
internal  auditors  and  the  work  force,  and  consultant  support  averaged  $250,000  to 
$300,000  for  the  pilot  sites. 

Effect  on  Organizational  Structure 

While  the  organizations  undergoing  the  ISO  certification  process  experienced  a 
reduction  in  the  number  of  full-time  equivalent  personnel  (FTEs),  their  basic 
structure  did  not  change.  Implementation  of  ISO  standards  and  the  certification 
process  did  not  cause  the  FTE  reductions  nor  are  any  major  reorganizations  con¬ 
sidered  as  a  result  of  the  ISO  9000  program.  (Louisville  Engineering  moved  the 
quality  assurance  manager  from  the  Engineering  Management  Branch  to  report 
directly  to  the  assistant  chief.) 

ISO  Consultant  Requirements 

The  USAGE  pilot  organizations  relied  heavily  on  ISO  consultants  during  the  certi¬ 
fication  process.  However,  the  consultants  were  unfamiliar  with  engineering  and 
construction  activities.  The  respondent  organizations  felt  that  the  value  of  die  ISO 
consultants  would  be  enhanced  if  they  understood  how  the  standards  apply  to  the 
USAGE  design  and  construction  business.  Since  the  ISO  consultants  supporting 
their  ISO  9000  program  had  limited  knowledge  of  these  Gorps  activities,  the  gap 
analysis  report  and  associated  briefing  were  not  as  effective  as  they  could  have 

’  The  estimated  12-to  18-month  time  frame  assumes  6  to  12  months  for  system  preparation 
and  6  months  to  implement  and  operate  the  quality  management  system. 
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been.  The  consultants  did  not  address  the  intent  of  the  standards’  elements  and 
their  relationship  to  Corps  business. 

Not  only  would  having  consultants  who  are  familiar  with  engineering  and  con¬ 
struction  activities  be  beneficial  to  the  Corps,  but  using  the  same  consultants  for 
each  new  district  seeking  ISO  9000  registration  also  would  be  beneficial.  The 
consultants  could  share  the  experiences  of  the  districts  that  have  already  attained 
registration. 

ISO  9000  and  Existing  USAGE  Quality  Requirements 

Concern  that  ISO  9000  requirements  might  be  incompatible  with  existing  US  ACE 
quality  requirements — such  as  the  quality  assurance  role  of  the  major  subordinate 
commands  (MSCs) — was  largely  unfounded.  While  nuances  in  the  terminology 
exist,  attention  to  detail  on  the  part  of  employees  involved  in  the  documentation 
of  procedures  appears  sufficient  to  overcome  this  problem.  It  might  be  useful, 
however,  to  produce  a  glossary  of  compatible  terminology  based  on  the  experi¬ 
ence  of  participating  districts. 

Simultaneous  Registration  of  Multiple  District  Organizations 

The  pilot  sites  were  intentionally  planned  to  pursue  ISO  9000  certification  one 
division  at  a  time.  The  thinking  was  to  test  the  applicability  of  the  ISO  9000  man¬ 
agement  system  within  the  purview  of  a  single  senior-level  manager  (one  divi¬ 
sion)  rather  than  undertake  a  broader  and  more  complex  assignment  dealing  with 
multiple  managers  (stovepipes). 

If  multiple  organizations  within  a  district  were  to  undergo  the  ISO  registration 
process  at  the  same  time,  the  district  engineer  would  have  to  make  an  ongoing 
commitment  of  resources  and  support.  The  management  representative  would 
have  to  be  selected  from  within  the  executive  office  for  the  successful  registration 
of  more  than  one  division  under  one  certificate.  One  advantage  of  multiple  certifi¬ 
cation  cited  was  that  this  approach  would  assist  in  the  breakdown  of  organiza¬ 
tional  stovepipes.  These  stovepipes  have  historically  impeded  coordination  among 
divisions  and  taken  a  toll  on  organizational  efficiencies  and  customer  support. 
Another  advantage  is  cost.  Leading  two  or  more  district  organizations  through  the 
process  simultaneously,  rather  than  sequentially,  would  produce  efficiencies 
throughout  the  planning,  organizing,  and  implementation  phases.  Could  a  district 
do  this  without  negatively  impacting  ongoing  core  activities?  This  is  a  key  ques¬ 
tion  to  consider  before  embarking  on  such  an  approach.  The  pilot  sites  generally 
agreed  that  the  least  disruptive  approach  would  be  to  continue  to  proceed  one  di¬ 
vision  at  a  time. 
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The  issue  of  costs  and  benefits  is  of  interest  to  all  parties.  Unfortunately,  no  quan¬ 
tifiable  data  are  available  to  analyze  costs  and  benefits  in  a  rigorous  financial 
framework.  We  do,  however,  have  strong  evidence  that  a  significantly  favorable 
cost-benefit  exists.  All  pilot  sites  stated  that  operating  under  the  ISO  9000  quality 
management  system  has  had  a  demonstrably  beneficial  effect  on  the  Corps  and 
that  they  would  recommend  ISO  9000  registration  to  other  districts.  The  districts 
that  participated  in  the  pilot  program  cited  a  variety  of  benefits,  the  most  promi¬ 
nent  being  a  divisionwide  clarification  of  procedures  and  the  genesis  of  an  under¬ 
standable,  well-defined  operating  system. 

Participating  organizations  have  not  operated  under  the  ISO  9000  standards  long 
enough  to  identify  and  quantify  tangible  benefits  and  savings.  However,  they  be¬ 
lieve  that  going  through  the  ISO  registration  process  has  increased  their  under¬ 
standing  of  their  business  processes  and  has  heightened  their  awareness  of  quality 
and  its  importance  to  meeting  customer  expectations.  They  also  point  to  the  sig¬ 
nificant  costs  savings  and  other  benefits  they  have  achieved  by  using  the  intranet 
to  make  all  technical  references  readily  available,  minimizing  the  need  for  hard 
copies,  and  ensuring  the  use  of  the  most  current  documents.  Putting  all  reference 
regulations,  codes,  and  standards  on  the  intranet  is  a  significant  cost  savings,  par¬ 
ticularly  for  tile  construction  divisions,  and  has  dramatically  improved  their  ability 
to  quickly  access  technical  information. 

Participants  have  pointed  out  that  the  costs  they  incurred  on  ISO  registration 
would  have  been  incurred  for  something  similar  anyway;  each  division  has  a  sen¬ 
ior  management  staff  that  spends  a  considerable  amount  of  time  on  quality  poli¬ 
cies,  procedures,  and  issues.  Deciding  to  spend  their  funds  specifically  on 
becoming  compliant  with  ISO  9000  was  just  a  matter  of  setting  priorities  on  how 
to  spend  the  discretionary  portion  of  the  departmental  overhead  budget. 

To  provide  a  glimpse  of  the  type  of  quantifiable  data  that  may  become  available 
concerning  the  costs  and  benefits  of  ISO  9000,  we  will  use  the  experience  of 
Louisville  Engineering.  Over  several  quarters,  that  division  reduced  its  projected 
cost  growth  (due  to  errors,  changes,  omissions,  etc.)  by  50  percent.  Although 
many  variables  likely  contributed  to  the  reduction,  certainly  some  of  this  impres¬ 
sive  reduction  can  be  attributed  to  the  division’s  successful  implementation  of  the 
ISO  9000  quality  system. 

The  potential  impact  of  implementing  an  ISO  quality  management  system 
throughout  the  Corps  can  be  illustrated  by  the  following  example: 

Assume  the  Corps’  construction  portfolio  of  $2,160  million  includes  a 
total  cost  growth  of  6.15  percent,  or  $133  million.  If,  by  operating  under 
an  ISO  quality  management  system,  the  Corps  could  reduce  the  cost 
growth  to  3  percent,  it  could  save  an  estimated  $66  million  per  year. 
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In  addition  to  the  information  gained  from  surveying  the  experience  of  the  pilot 
sites,  we  have  looked  at  the  available  literature  to  learn  from  the  private-sector 
experience  with  ISO  9000.  In  1996,  Dun  and  Bradstreet  conducted  a  nationwide 
survey  of  organizations  registered  to  ISO  9000;  1,880  organizations  responded. 
The  results  are  summarized  in  Table  4-1. 


Table  4-1.  Responses  to  Dun  and  Bradstreet  Survey 


Factor 

Response 

Higher  perceived  quality 

83% 

Competitive  advantage 

70% 

Reduced  customer  quality  audits 

56% 

Improved  customer  demand 

29% 

Increased  market  share 

18% 

Quicker  delivery  of  products 

6% 

Better  documentation  as  an  internal  benefit 

88% 

Greater  quality  awareness  by  employees 

83% 

Enhanced  internal  communications 

53% 

Increased  operational  efficiency 

40% 

Main  driver  for  seeking  ISO  9000  certification — quality  benefits 

77% 

Main  driver  for  seeking  ISO  9000  certification — market  advantage 

73% 

Main  driver  for  seeking  ISO  9000  certification — customer  expectation 

68% 

Average  total  cost  of  registration 

$187,000 

Average  total  cost  of  registration:  1 993 

$245,000 

Source:  Irwin  Professional  Publishing  and  Dun  and  Bradstreet  Information  Services,  ISO 
9000  Survey:  Comprehensive  Data  and  Analysis  of  U.S.  Registered  Companies,  1 996. 

Note:  Percentages  are  rounded. 


The  bottom  line  on  costs  versus  benefits  is  that,  even  in  the  absence  of  reams  of 
quantifiable  data,  it  is  abundantly  clear  that  USAGE  will  benefit  significantly  as 
more  and  more  of  its  organizations  receive  their  ISO  9000  certificates  of  registra¬ 
tion. 
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Conclusions  and  Recommendations 


The  organizations  participating  in  the  pilot  program  are  enfliusiastic  about  both 
the  experience  and  the  results  of  the  ISO  9000  certification  process.  From  then- 
comments  and  our  observations,  we  conclude  that  the  benefits  of  ISO  certification 
outweigh  the  costs  in  time  and  money. 

ISO  9000  registration  undoubtedly  offers  valuable  benefits  to  the  Corps.  Pilot  or¬ 
ganizations  report  that  their  postregistration  operations  are  streamlined,  more  effi¬ 
cient,  and  consistent  with  the  operation  of  an  engineering  and  construction 
organization.  The  rigor  of  maintaining  registration  is  of  value  as  well;  it  forces 
organizational  self-examination  in  preparation  for  external  audits.  Registration  is 
just  the  initial  step;  the  positive  effects  of  operating  under  an  ISO  9000  quahty 
system  can  be  expected  to  continue  indefinitely,  not  just  inunediately  after  imple¬ 
mentation. 

Less  tangible,  but  equally  worthy,  is  the  effect  of  ISO  9000  on  morale.  ISO  9000 
represents  a  positive  cultural  change.  At  all  pilot  sites  it  has  led  to  better  internal 
communication,  increased  teamwork,  improved  the  ability  of  individuals  to  see 
the  big  picture,  reduced  the  time  required  to  deal  with  routine  procedures,  clarified 
responsibilities,  and  reduced  errors  and  rework.  The  benefits  of  operating  under 
the  ISO  quality  management  system  are  undeniable. 

Some  divisions  viewed  the  cost  of  the  ISO  9000  registration  process  as  too  high. 
However,  with  the  benefits  of  improved  efficiency  and  effectiveness,  we  believe 
these  costs  are  reasonable  given  anticipated  time  and  labor  savings.  In  any  case, 
some  management  time  must  be  spent  in  the  quality  policies  and  procedures 
arena,  so  some  of  the  cost  incurred  would  have  been  necessary  even  without  ISO 
9000  implementation.  In  short,  Corps-wide  ISO  9000  registration  appears  to  be  a 
worthy  strategy  for  ensuring  that  USAGE  consistently  delivers  quality  products 
and  services. 

To  facilitate  the  implementation  of  ISO  9000  throughout  the  Corps,  we  recom¬ 
mend  that  USAGE  take  the  following  steps: 

♦  Adopt  ISO  9000  as  the  Corps  quality  management  standard.  Revise  ER 

1 1 10-1-12  accordingly  to  direct  the  application  of  the  standard  and  main¬ 
tenance  of  registration. 

♦  Direct  all  engineering  and  construction  divisions  to  achieve  and  maintain 
registration  to  the  ISO  9000  standard.  The  pilot  sites  clearly  benefited 
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from  preparing  for,  achieving,  and  maintaining  their  registration  to  the  ISO 
quality  management  system. 

♦  Develop  customized  training  materials.  Each  division  spent  a  great  deal  of 
time  organizing  its  efforts  before  it  began  defining  its  processes  and 
documenting  them.  The  organization  phase  could  be  streamlined  through 
the  creation  of  a  customized  training  curriculum  made  available  to  all  divi¬ 
sions.  It  would  be  relatively  easy  to  produce  a  Corps-specific  set  of  guide¬ 
lines  and  implementation  recommendations.  Once  these  guidelines  and 
recommendations  were  in  place,  USAGE  districts  could  expedite  achiev¬ 
ing  ISO  9000  registration. 

♦  Create  a  model  policy  and  procedures  manual.  To  decrease  costs  associ¬ 
ated  with  preparing  the  policy  and  procedures  manual,  we  recommend  that 
the  quality  manuals  created  by  the  pilot  sites  be  used  as  the  basis  for  cre¬ 
ating  a  model  policy  and  procedures  manual  that  could  be  applied  in  a 
generalized  fashion  to  all  USAGE  ISO  9000  registration  efforts  and  fully 
utilize  knowledge  gained  from  the  ISO  9000  implementation  process. 
Having  a  model  manual  would  make  it  significantly  easier  for  districts  to 
get  started.  However,  the  model  manual  would  not  be  a  substitute  for  the 
hard  work  of  each  organization  examining  its  business  processes.  Private- 
sector  companies  with  multiple  sites  have  used  this  approach  successfully. 

♦  Form  a  headquarters  working  group.  The  ISO  9000  management  system 
offers  many  opportunities  to  improve  coordination,  not  only  within  the 
districts  but  also  at  all  levels  of  the  Gorps.  We  recommend  forming  an  ISO 
9000  working  group  at  headquarters  to  be  the  nerve  center  and  clearing¬ 
house  for  ISO  experience  within  the  Gorps.  The  working  group  should  de¬ 
velop  a  detailed  implementation  strategy;  coordinate  ongoing  program 
activities;  compile  data  on  experience,  successful  strategies,  and  lessons 
learned;  communicate  with  USAGE  senior  management;  and  raise  issues 
for  management  resolution.  The  group  would  function  as  a  focal  point  for 
continuous  program  improvement. 

♦  Develop  annual  experience  evaluation  metrics.  A  benefit  of  an  ISO  9000 
operating  system  is  its  ready  translation  to  usage  as  an  evaluation  metric. 
ISO  9000  can  serve  as  a  basis  on  which  to  judge  performance,  a  usage  that 
fits  well  with  usage’s  vision  of  continuous  improvement.  We  reeom- 
mend  development  of  performance  metrics  and  continuous  improvement 
measures  to  track  progress  at  each  division  that  achieves  certification.  For 
example,  targets  for  measuring  the  ongoing  system  suitability  and  effec¬ 
tiveness  should  be  developed.  Internal  measures  could  include  aspects  re¬ 
lated  to  projeet  delivery,  system  audits,  reviews,  corrective  and  preventive 
actions,  and  predelivery  design  changes.  External  measures  might  cover 
customer  reviews,  cost  and  schedule  performance,  customer  feedback,  and 
postdelivery  modifications. 
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Conclusions  and  Recommendations 


♦  Procure  a  single  registrar.  We  recommend  that  headquarters  acquire  and 
make  available  a  task  order  contract  for  registrar  services.  This  approach 
would  save  the  districts  both  the  time  and  cost  of  procuring  those  services, 
and  the  total  cost  to  the  Corps  of  ordering  registrar  services  off  a  master 
contract  would  be  less  overall  than  purchasing  them  individually.  Fur¬ 
thermore,  since  the  registrar  plays  such  an  important  educational  role  in 
the  process,  using  the  services  of  one  organization  familiar  with  USAGE 
and  its  operations  would  be  advantageous. 

♦  Begin  concurrently  implementing  ISO  quality  management  systems  in 
multiple  divisions  within  a  district.  Implementing  an  ISO  quality  manage¬ 
ment  system  in  multiple  district  divisions  (programs,  planning,  engineer¬ 
ing,  construction,  and  contracts)  at  the  same  time  appears  to  be  a  feasible 
and  attractive  approach  that  the  Corps  should  test.  Although  defining  inter¬ 
faces  among  divisions  could  be  time-consuming  and  potentially  difficult, 
the  existence  of  US  ACE-specific  training  materials  and  a  model  quality 
manual  would  ease  the  process  considerably.  Implementation  would  likely 
become  the  only  area  requiring  a  serious  time  commitment  on  the  part  of 
individual  districts  and  their  respective  divisions. 

♦  Test  application  of  ISO  9000  beyond  the  district  level.  Headquarters  and 
the  major  subordinate  commands  could  benefit  from  ISO  9000  registra¬ 
tion;  the  ISO  quality  management  system  should  be  applied  throughout  the 
Corps,  not  just  at  the  district  level.  Should  the  Corps  decide  to  implement 
this  recommendation,  it  would  need  to  evaluate  existing  directives,  guid¬ 
ance,  management  protocols,  and  regulations  in  the  context  of  an  effective 
Corps- wide  ISO  9000  program.  Audit  requirements  for  the  MSCs  and  dis¬ 
tricts  would  also  have  to  be  reconsidered.  In  the  meantime,  USAGE  head¬ 
quarters  should  evaluate  its  directives  and  guidance  documents  in  terms  of 
their  effects  on  individual  units  that  achieve  ISO  certification. 

♦  Train  major  subordinate  command  quality  assurance  people  as  internal 
auditors.  MSC  quality  assurance  people  should  be  trained  as  ISO-certified 
internal  auditors.  Under  tiieir  new  role  as  “business  centers,”  the  MSCs 
should  also  be  certified  to  ISO  9000  and  can  use  the  process  of  preparing 
for  registration  to  work  through  and  define  their  new  business  processes. 
The  quality  assurance  people  could  also  play  a  productive  role  in  assisting 
districts  with  the  registration. 

♦  Apply  electronic  and  Web  technology  for  document  control.  Key  to  the 
success  of  the  engineering  and  construction  divisions  is  the  use  of  elec¬ 
tronic  document  control  strategies  and  the  use  of  the  Corps  intranet  to  give 
all  offices  access  to  reference  documents.  The  work  that  has  gone  into  this 
in  Louisville  and  Savannah  should  be  utilized  to  develop  a  generic  docu¬ 
ment  control  system.  Such  a  tool  would  save  time  and  money  for  any 
USAGE  organization  pursuing  ISO  registration. 
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Appendix  A 

Guidelines  for  USAGE  Conformance 
with  ISO  9000  Standards 


Introduction 

This  appendix  provides  USAGE  an  interpretation  of  the  ISO  9000  quality  system 
standards  as  they  apply  specifically  to  managing  USAGE  engineering  and  design 
(E&D)  and  construction  projects,  including  civil  works;  military  construction;  or 
hazardous,  toxic,  and  radioactive  waste  projects.  It  also  contains  guidance  for 
complying  with  die  requirements  of  the  standards  in  terms  that  should  be  familiar 
to  typical  USAGE  engineering  and  construction  personnel.  The  interpretation  and 
guidance  in  this  appendix  should  not  be  applied  to  other  USAGE  district  functions 
such  as  project  management,  contracting,  and  resource  management. 

This  appendix  should  be  read  in  conjunction  with  the  ISO  9000  standards  (ISO 
9001  for  engineering  activities  and  ISO  9002  for  construction  activities).*  The  re¬ 
quirements  in  the  ISO  standards  take  precedence  over  the  interpretation  provided 
in  this  document,  and  deviations  from  the  requirements  specified  in  the  ISO  stan¬ 
dards  must  be  sufficiently  addressed  in  the  organization’s  quality  policy  docu¬ 
mentation. 

References 

The  following  publications  describe  the  ISO  9000  quality  management  system: 

♦  ISO  9001,  Quality  Systems — ^Model  for  Quality  Assurance  in  Design, 
Development,  P’roduction,  Installation,  and  Servicing 

♦  ISO  9002,  Quality  Systems — ^Model  for  Quality  Assurance  in  Production, 
Installation,  and  Servicing 

♦  ISO  9003,  Quality  Systems — ^Model  for  Quality  Assurance  in  Final  In¬ 
spection  and  Test 

♦  ISO  9004,  Parts  1-4,  Quality  Management  and  Quality  System  Elements 
(general  guidelines  and  guidelines  for  services,  processed  materials,  and 
quality  improvement) 


*  ISO  9000  standards  are  reproduced  in  the  United  States  under  the  American  National  Stan¬ 
dards  Institute  and  the  American  Society  of  Quality  Control  Q9000  series. 
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♦  ISO  8402,  Quality  Management  and  Quality  Assuremce — ^Vocabulary 

♦  ISO  Guide  34,  Quality  System  Guidelines  for  the  Production  of  Reference 
Materials 

♦  ISO  9000,  Parts  1-4,  Quality  Management  and  Quality  Assurance  Stan¬ 
dards  (guidelines  for  selection,  use,  and  application  of  ISO  9001,  ISO 
9002,  and  ISO  9003) 

♦  ISO  10005,  Quality  Management — Guidelines  for  Quality  Plans 

♦  ISO  10006,  Quality  Management — Guidelines  to  Quality  in  Project  Man¬ 
agement 

♦  ISO  10007,  Quality  Management — Guidelines  for  Configuration  Man¬ 
agement 

♦  ISO  10011,  Parts  1-3,  Guidelines  for  Auditing  Quality  Systems  (auditing, 
qualification  criteria  for  auditors,  and  management  of  audit  programs) 

♦  ISO  10012,  Parts  1-2,  Quality  Assurance  Requirements  for  Measuring 
Equipment  (metrological  confirmation  system  and  guidelines  for  control¬ 
ling  measurement  processes) 

♦  ISO  10013,  Guidelines  for  Developing  Quality  Manuals 

Relevant  USAGE  publications  about  engineering  and  construction  management 
include  the  following: 

♦  Army  Regulation  5-1,  Army  Management  Philosophy 

♦  Engineering  Publication  (EP)  415-1-260,  Resident  Engineer  Management 
Guide 

♦  EP  415-1-266,  Resident  Engineer  Management  guide  for  Hazardous, 
Toxic,  and  Radioactive  Waste  Projects 

♦  Engineering  Regulation  (ER)  1-1-11,  Administration — ^Progress,  Sched¬ 
ules,  and  Network  Analysis  Systems 

♦  ER  25-345-1,  System  Operation  and  Maintenance  Documentation 

♦  ER  415-1-10,  Contractor  Submittal  Procedures 

♦  ER  415-1-11,  Biddability,  Constructibility,  Operability,  and  Environ¬ 
mental 

♦  ER  415-1-13,  Design  and  Construction  Evaluation  (DCE) 
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Guidelines  for  USAGE  Conformance  with  ISO  9M0  Standards 


♦  ER  415-3-11,  Engineering  and  Design  Post-Completion  Inspection  Feed¬ 
back 

♦  ER  415-7-1,  Contractor  Performance  Evaluations 

♦  ER  715-1-8,  Architect-Engineer  Contract  Administration  Support  System 

♦  ER  715-1-10,  Architect  Engineers  Responsibility  Management  Program 

♦  ER  715-1-15,  Time  Standards  for  the  Architect  Engineer  Acquisition 
Process 

♦  ER  1 1 10-1-12,  Quality  Management 

♦  ER  1 1 10-1-263,  Chemical  Quality  Management  for  Hazardous  Waste 
Remedial  Activities 

♦  ER  1 1 10-1-1 150,  Engineering  and  Design  for  Civil  Works  Projects 

♦  ER  1 1 10-1-1200,  Drawings  and  Specifications 

♦  ER  1 1 10-345-100,  Design  Policy  for  Military  Construction 

♦  ER  1 1 1 0-345 -700,  Design  Analysis 

♦  ER  111 0-345-7 1 0,  Drawings 

♦  ER  1 1 10-345-720,  Construction  Specifications 

♦  ER  1180-1-6,  Contracts — Construction  Quality  Management. 

Definitions 

The  ISO  9000  standards  carry  a  specific  set  of  definitions  consistently  applied 
throughout  the  series  of  contractual  and  guidance  documents.  Those  definitions, 
further  defined  as  they  specifically  apply  to  US  ACE  engineering  and  construction 
organizations,  are  as  follows: 

♦  Supplier — ^the  US  ACE  engineering  or  construction  division  (or  other  or¬ 
ganizational  activity  with  similar  responsibilities). 

♦  Product — ^the  result  of  activities  or  processes.  The  ISO  9000  standards 
note  that  a  product  may  be  a  service  rendered,  hardware,  processed 
materials,  software,  or  any  combination  thereof,  and  it  may  be  tangible  or 
intangible  (e.g.,  knowledge  or  concepts).  For  USACE  engineering 
organizations,  the  product  can  be  the  engineering  plans  and  specifications, 
engineering  studies,  and/or  services  provided  that  relate  to  the 


A-3 


management  and  support  of  any  E&D  project  throughout  its  life  cycle 
(project  initiation  and  planning,  design  development,  design  support  and 
servicing).  For  USAGE  construction  activities,  the  product  is  a  service — 
management  of  a  construction  project  throughout  its  life  cycle  (scheduling, 
quality  assurance,  cost  control,  claims  analysis). 

♦  Customer — anyone  or  any  organization  that  receives  a  product  from  a 
USAGE  engineering  or  construction  division.  Customers  can  be  either  in¬ 
ternal  or  external  to  die  USAGE  district.  Internal  customers  may  include 
the  district’s  project  management,  contracting,  construction,  or  operations 
activities,  while  external  customers  may  include  Army  installations.  Air 
Force  installations,  or  other  federal  or  state  governmental  agencies. 

♦  Contract — any  contract  instrument  or  medium  (written  or  oral)  used  to  es¬ 
tablish  agreed-upon  requirements  between  the  USAGE  supplier  and  the 
customer.  The  contract  may  be  a  memorandum  of  agreement  (MOA), 
memorandum  of  understanding  (MOU),  delivery  order,  DD  Form  1391, 
work  order,  action  required  under  the  project  management  plan,  or  tradi¬ 
tional  contract  vehicle. 

♦  Tender — offer  made  by  the  engineering  or  construction  organization  in  re¬ 
sponse  to  any  invitation  by  a  customer  for  the  provision  of  a  product. 

♦  Prime  contractor — any  firm  hired  by  USAGE  to  execute  a  construction 
contract  (referred  to  as  subcontractor  in  the  ISO  standard). 

♦  Subcontractor — any  firm  hired  by  the  USAGE  district  to  provide  A-E 
services,  laboratory  services,  or  construction  services. 

♦  Government-furnished  equipment  (GEE) — all  materials  either  purchased 
by  the  USAGE  district’s  contracting  activity  or  by  the  customer  for  inclu¬ 
sion  in  the  contracted  facility.  Examples  include  fixtures,  furnishings,  and 
specialized  equipment  that  a  customer  may  provide  under  another  contract 
that  will  be  delivered  to  the  site  for  placement  or  installation  in  the  con¬ 
tracted  facility. 

Quality  System  Requirements 

The  following  subsections  provide  the  requirements  and  guidance  for  the  20  ele¬ 
ments  that  a  USAGE  engineering  division  must  address  to  comply  with  ISO  9001. 
The  numbering  structure  coincides  exactly  with  that  used  in  the  ISO  9001  stan¬ 
dard.  A  USAGE  construction  division  must  address  all  of  the  same  elements,  with 
one  exception — 4.4,  Design  Gontrol. 
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Guidelines  for  USAGE  Conformance  with  ISO  9000  Standards 


4.1  Management  Responsibility 

4.1.1  Quality  Policy 
Requirement 


To  meet  the  requirements  of  this  element,  senior  management  (chief,  deputy,  or 
committee  of  branch  chiefs)  must  make  siu^  that 

♦  a  quality  policy  and  quality  objectives  are  well  defined,  approved,  audit- 
able,  measurable,  and  sufficiently  documented; 

♦  the  policy  cites  commitment  to  developing  the  highest  quality  products, 
meeting  customers’  expectations  for  quality  products  and  services,  and 
satisfying  personnel  needs; 

♦  everyone  in  the  organization  understands,  embraces,  and  practices  what  is 
represented  by  the  policy;  and 

♦  the  quality  policy  is  consistent  with  USAGE  and  district  quality  directives, 
engineering  and  construction  policies,  organizational  quality  goals,  and  the 
expectations  of  customers. 

Note:  Quality  policy  is  defined  by  ISO  8402  as  “the  overall  intentions  and  direc¬ 
tion  of  an  organization  with  regard  to  quality,  as  formally  expressed  by  top  man¬ 
agement.”  ISO  9004  defines  quality  management  as  “that  aspect  of  the  overall 
management  function  which  determines  and  implements  quality  policy.”  The  ele¬ 
ment  emphasizes  that  the  responsibility  for  quality  belongs  at  the  highest  level  of 
management  in  an  organization. 


Guidance 


The  quality  policy  and  objectives  developed  and  promulgated  by  the  engineering 
or  construction  division  should  be  consistent  with  any  policies,  goals,  and  mission 
statements  already  in  place  in  the  district.  Moreover,  they  should  be 

♦  relevant,  measurable,  and  ambitious,  yet  achievable,  and 

♦  easy  for  everyone  in  the  division  to  understand. 

The  quality  policy  and  objectives  should  be  specific.  It  may  not  be  enough  simply 
to  say  that  the  division  will  reduce  design  costs,  but  rather  that  they  will  be  re¬ 
duced,  for  example,  by  at  least  10  percent.  The  following  are  examples  of  quality 
objectives  tiiat  are  relevant,  measurable,  and  achievable: 

♦  Design  projects  will  be  completed  on  time  95  percent  of  the  time  (on  time 
means  ready  to  advertise  within  1  month  of  original  scheduled  date). 
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♦  Construction  projects  will  be  completed  within  budget  95  percent  of  the 
time  (within  budget  means  not  exceeding  the  programmed  amount  by  10 
percent). 

♦  Controlled  design  cost  growth  will  not  exceed  2  percent. 

♦  All  contract  actions  will  be  closed  out  within  6  months  of  owner  occu¬ 
pancy. 

In  addition,  while  not  required  specifically  by  the  standard,  the  division’s  execu¬ 
tive  management  should  do  the  following: 

♦  Define  its  key  elements  of  quality,  such  as  fitness  for  use,  responsiveness 
to  customers,  effective  cost  control,  and  compliance  with  the  organiza¬ 
tion’s  internal  policy  and  regulations,  guide  specifications,  and  relevant 
codes. 

♦  Consider  the  ramifications  and  costs  if  its  quality  objectives  are  not  met. 

♦  Provide  sufficient  resources  to  develop  the  organization’s  policies  and 
objectives  so  that  they  are  meaningful,  measurable,  achievable,  and  under¬ 
standable  to  everyone  in  the  engineering  organization  (it  is  good  practice 
to  have  the  staff  participate  in  developing  them). 

♦  Determine  the  required  skill,  experience,  and  training  to  achieve  those 
goals. 

♦  Ensure  that  the  policy  and  objectives  are  written  simply  and  clearly. 

4.1.2  Organization 

4. 1.2.1  Responsibility  and  Authority 

Requirement 


The  engineering  or  construction  division  must  define  the  responsibiUties,  authori¬ 
ties,  and  interrelationships  of  everyone  managing  or  otherwise  affecting  the  qual¬ 
ity  of  its  products  and  services.  The  documentation  must  include  organization 
charts  along  with  functional  descriptions  for  branches,  sections,  and  field  activi¬ 
ties  under  its  authority.  In  addition,  the  documentation  must  refer  specifically  to 
the  personnel  who  review  and  verify  products  as  well  as  correct  or  prevent  non¬ 
conformities  occurring  in  the  product,  system  procedures,  or  quality  system. 
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Guidance 

Members  at  all  levels  of  the  division  should 

♦  understand  their  job  functions  and  responsibilities, 

♦  understand  for  what  responsibilities  and  actions  they  wiU  be  held  account¬ 
able, 

♦  be  aware  of  their  impact  on  product  and  service  quality, 

♦  have  adequate  authority  to  carry  out  their  responsibilities  in  relation  to 
quality,  and 

♦  accept  responsibility  for  achieving  quality  objectives. 

4. 1.2.2  Resources 
Requirement 


The  division’s  senior  management  must  make  sure  that  sufficient  resources  are 
available  for  quality  control  and  quality  assurance  activities  on  every  in-house 
project  so  that  completed  products  conform  to  stated  requirements.  In  addition, 
personnel  assigned  to  each  project  must  be  adequately  trained  and  possess  the  ex¬ 
perience  and  skills  to  successfully  carry  out  their  job  assignments  (on-the-job 
training  is  permissible).  See  4.18,  Training.  Senior  management  also  must  make 
sure  that  adequate  resources  are  available  for 

♦  inspection  and  verification  of  work  performed  by  contractors,  subcontrac¬ 
tors,  or  engineering  laboratories; 

♦  review  and  approval  of  quality  control  plans  submitted  by  contractors; 

♦  in-process  reviews  and  verification  of  contractor  quality  control  plans;  and 

♦  internal  quality  system  audits. 


Guidance 


Effective  verification  of  the  products  requires  cooperation  and  objectivity  among 
everyone  involved.  Adequate  resources  can  involve  the  following: 

♦  Awareness  of  the  ISO  standards  and  the  organization’s  quality  system 

♦  Adequate  training  to  perform  the  assigned  duties 

♦  Realistic  quality  assurance  plans  that  allow  ample  resources  for  project 
scope  development  as  well  as  design  reviews  and  verification;  biddability. 
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constructibility,  operability,  and  environmental  (BCOE)  reviews;  supervi¬ 
sion;  and  required  engineering  tests 

♦  Adequate  and  proper  equipment  (e.g.,  computers,  CD-ROM,  computer- 
aided  design  and  drafting  workstations),  software,  and  other  resources  to 
perform  any  quality  system  assignment. 

Frequently,  training  budgets  are  set  at  (he  USAGE  district  level  and  are  beyond 
the  control  of  engineering  or  construction  management.  In  such  cases,  manage¬ 
ment  should  demonstrate  that  training  requirements  are  identified  and  priorities 
established  and  that  the  high  priorities  are  funded.  Also,  management  can  demon¬ 
strate  a  commitment  to  training  if  it  redirects  engineering  or  construction  division 
funds  to  meet  those  training  priorities. 

4. 1.2.3  Management  Representative  Requirement 

Requirement 


The  division’s  senior  management  must  appoint  a  management  representative 
who  has  the  responsibility  and  authority  for  establishing  and  maintaining  the 
quality  system  and  for  reporting  to  senior  management  on  that  systeni  for  reviews 
on  system  performance  and  opportunities  for  improvement.  The  appointed  man¬ 
agement  representative  must  be  a  member  of  the  management  team,  must  be  offi¬ 
cially  recognized,  and  must  be  identified  in  the  quality  manual  or  similar 
documentation. 


Guidance 


Anyone  in  the  division  could  be  selected  as  the  management  representative — the 
chief,  the  deputy,  any  one  of  the  branch  chiefs,  or  any  other  individual  who  is 
given  the  required  organizational  authority  and  line  of  control  to  the  chief.  The 
management  representative  should  also  have  representation  at  any  field  activity  by 
direct  line  of  authority  or  through  the  appointment  of  agents  at  each  field  activity. 
A  management  representative  selected  from  the  division’s  management  team  wUl 
already  have  the  needed  authority  to  carry  out  the  responsibilities  associated  with 
the  position.  That  person  should  also  possess  the  following  qualifications: 

♦  Be  knowledgeable  about  traditional  quality  assurance  techniques 

♦  Thoroughly  understand  the  ISO  9000  quality  system  standards 

♦  Be  conmiitted  to  the  importance  of  ISO  9000  to  the  engineering  organiza¬ 
tion 

♦  Be  well  respected  within  the  organization 

♦  Have  excellent  communication  skills. 
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If  the  person  selected  to  be  the  management  representative  does  not  already  have 
the  required  quality  system  knowledge,  training  will  be  necessary. 

The  management  representative  may  have  other  responsibilities  in  addition  to 
quality  system  oversight,  but  those  other  duties  should  pose  no  conflict  of  interest 
with  the  duties  associated  with  quality  system  oversight.  Moreover,  the  person’s 
role  as  management  representative  should  be  at  least  as  important  as  those  other 
duties. 

4.1.3  Management  Review 
Requirement 


At  defined  and  regular  intervals,  the  division’s  senior  managers  must  evaluate  the 
existing  quality  system  to  ensure  that  it  continues  to  be  suitable  and  effective  and 
to  resolve  any  outstanding  issues  relating  to  the  quality  system  and  stated  quality 
policy,  goals,  and  objectives.  Meeting  minutes  and  actions  taken  must  be  recorded 
and  maintained. 


Guidance 


The  frequency  of  management  reviews  is  not  specified  in  the  standard  and  will 
depend  on  local  circumstances.  Typically,  however,  auditors  expect  to  see  records 
of  progress  since  the  last  audit.  Since  audits  occur  at  least  annually  (for  any  par¬ 
ticular  element),  it  is  widely  held  that  management  reviews  should  be  scheduled  at 
least  as  often  as  the  registrar’s  ongoing  annual  or  biannual  assessments.  In  terms 
of  foUow-up,  problems  should  be  documented,  analyzed,  and  resolved  in  a  timely 
manner.  The  quality  system  reviews  should  address  the  following  questions: 

♦  Is  the  quality  system  (still)  working  effectively? 

♦  Is  the  organization  achieving  its  measurable  objectives  and  stated  quality 
policy  (see  4.1.1,  Quality  Policy)? 

♦  Are  documented  procedures  current  and  consistent  with  the  way  they  are 
applied  by  staff? 

While  the  scope  of  the  review  rests  with  the  division’s  senior  managers,  manage¬ 
ment  should  review  at  least  the  following  elements: 


♦  Results  of  internal  quality  system  audits  (see  4.17,  Internal  Quality  Audits) 

♦  Summaries  of  system  nonconformities  and  deficiencies  and  of  corrective 
and  preventive  actions  taken  (see  4.14,  Corrective  and  Preventive  Action) 

♦  Effectiveness  of  current  organizational  structure  and  current  quality  infi-a- 
structure 
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♦  Progress  toward  implementation  of  the  quality  system  and  attainment  of 
established  quality  goals  and  objectives  (revision  of  policies,  goals,  and 
objectives  may  be  necessary) 

♦  Current  and  required  training  and  effectiveness  of  training  taken 

♦  Perceived  and  achieved  quality  of  the  products  or  services 

♦  Information  from  customer  feedback. 

Guideunes  for  Conformity 

□  Within  the  organization,  has  senior  management  established  and  approved 
quality  policies  and  objectives,  and  are  they  well  written? 

□  Does  the  quality  policy  address  organizational  goals  and  customer  expec¬ 
tations? 

□  Are  procedures  in  place  to  ensure  that  the  policy  and  objectives  are  effec¬ 
tively  communicated  and  understood  by  everyone  in  the  organization? 

□  Has  management’s  commitment  to  those  policies  and  objectives  been  de¬ 
fined? 

□  Has  a  management  representative  within  the  organization  been  charged 
with  overall  responsibility  for  ensuring  that  all  ISO  9000  requirements  are 
implemented  and  maintained?  And  is  that  appointment  recorded  in  the 
quality  manual? 

□  Does  the  management  representative  have  sufficient  authority  to  develop, 
monitor,  and  change  the  quality  system?  Is  that  person  a  member  of  the 
management  team? 

□  Is  the  quality  system  periodically  reviewed  by  an  executive  committee  to 
ensure  that  it  is  still  suitable  and  effective,  and  are  records  of  those  reviews 
maintained? 

□  Are  persoimel  adequately  trained  in  the  organization’s  quality  management 
programs? 

□  Has  the  organization  defined  the  responsibilities  and  authorities  of  people 
involved  in  design  review  and  internal  quality  audits  as  well  as  people  in¬ 
volved  in  the  identification  and  recording  of  any  quality  system  or  product 
problems? 
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□  Are  organization  charts  and  functional  statements  documented,  and  do 
they  define  the  authority  and  responsibility  to  deal  with  design  process  and 
quality  system  problems? 

4.2  Quality  System 
4.2.1  General 
Requirement 


The  division  must  estabhsh,  document,  and  maintain  a  quahty  system  aimed  at 
conforming  to  its  customers’  specified  requirements  and  at  achieving  customer 
satisfaction.  That  documented  quality  system  must  include  a  quality  manual  (see 
guidance  below)  that  provides  Ae  organization’s  policy  for  each  of  the  relevant 
elements  of  the  ISO  quality  management  system,  as  well  as  Army,  USAGE,  dis¬ 
trict,  and  engineering  or  construction  policies  with  respect  to  those  ISO  elements. 
The  quality  manual  must  also  reference  quality  system  procedures  and  outline  the 
structure  of  the  quality  system.  Typically,  that  structure  includes  the  quality  man¬ 
ual;  quality  assurance  plans;  quality  system  procedures;  work  instructions;  other 
required  quality  system  documentation;  and  relevant  federal  legislation.  Army  and 
engineering  regulations,  technical  manuals,  and  guide  specifications. 

Note:  The  division  must  document  all  local  poUcies  and  practices  that  deviate 
from  Army  and  USAGE  regulations. 


Guidance 


The  quality  system  applies  to  all  activities  related  to  the  delivery  of  a  quality 
product.  These  activities  can  range  from  initial  project  development  through  de¬ 
sign  and  may  continue  through  construction  of  the  project  if  specified  in  the  con¬ 
tract. 

The  documented  quality  system  typically  has  a  tiered  structure,  with  each  tier  be¬ 
coming  increasingly  detailed.  At  the  top  tier,  the  quality  manual  should  explain 
the  division’s  policy  toward  each  element  of  the  ISO  standard  and  reference  the 
required  procedures.  At  the  second  tier,  documented  quality  system  procedures 
should  discuss  the  who,  what,  when,  and  where  for  each  applicable  element  and 
reference  appropriate  work  instructions;  that  documentation  should  be  short  and 
easily  understood,  and  the  procedures  should  be  illustrated  through  the  use  of  flow 
charts.  The  third  tier  defines  the  work  instructions — ^the  details  about  how  to  meet 
the  quality  system  requirements.  Quality  system  records  or  proof  of  compliance 
form  the  fourth  tier. 
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ISO  9004  stresses  that  the  goal  of  the  documented  quality  system  is  to  provide 
confidence  that 

♦  the  system  is  understood  by  everyone  in  the  organization  and  is  carried  out 
effectively, 

♦  the  product  satisfies  customer  and  industry  requirements  and  customer  ex¬ 
pectations, 

♦  the  needs  of  both  society  and  the  environment  have  been  addressed,  and 

♦  die  emphasis  is  on  problem  prevention  rather  than  detection  and  correction 
after  a  problem  has  occurred. 

The  quality  manual  may  include  die  following  elements: 

♦  Signature  of  the  division  chief  (or  designee) 

♦  Name  and  location  of  the  district  and  division 

♦  Organization’s  quality  policy,  goals,  and  measurable  objectives 

♦  Current  organization  charts  and  functional  statements 

♦  Designated  quality  responsibilities  and  name  of  the  management  repre¬ 
sentative 

♦  Accepted  management  review  system 

♦  Approved  methods  for  revising  and  updating  the  manual 

♦  Complete  coverage  of  all  quality  system  elements 

♦  List  of  quality  system  procedures. 

4.2.2  Quality  System  Procedures 
Requirement 


Documented  quality  system  procedures  that  address  the  relevant  ISO  9000  re¬ 
quirements  and  the  division’s  quality  policy,  goals,  and  objectives  must  be  effec¬ 
tively  implemented. 


Guidance 


Above  all  else,  the  documentation  of  the  quahty  system  proeedures  needs  to  be 
kept  simple;  two  to  four  pages  per  procedure  should  be  sufficient  under  most  con¬ 
ditions.  The  division  may  select  whatever  format  it  wants  for  its  procedures:  flow 
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or  process  charts,  models,  narrative,  or  a  combination  of  formats.  The  level  of 
detail  in  the  documented  procedures  should  be  inverse  to  the  level  of  training,  ex¬ 
perience,  and  qualifications  of  the  division’s  staff.  In  other  words,  relatively  little 
documentation  is  required  when  the  level  of  education,  experience,  and  qualifica¬ 
tions  of  die  staff  is  relatively  high.  Conversely,  for  a  staff  with  less  education,  ex¬ 
perience,  and  qualifications,  the  processes  and  procedures  should  be  documented 
in  greater  detail. 

4.2.3  Quality  Planing 

Requirement 


As  appropriate,  the  division  must  develop  an  overall  organizational  quality  man¬ 
agement  plan  that  shows  how  it  will  consistently  deliver  quality  products  and 
meet  its  stated  quality  policy  and  objectives.  It  must  also  develop  an  effective 
quality  assurance  plan  for  each  project.  The  plan  must  show  how  the  technical, 
schedule,  industry,  and  cost  requirements  for  the  project  will  be  met.  Quality  plans 
must  be  developed  in  accordance  with  USAGE  regulations,  division  policy,  and 
project-specific  requirements  outlined  in  the  project  management  plan  (see  4.9, 
Process  Control).  The  division  also  must  develop  guidance  that  shows  how  proj¬ 
ects  are  selected. 


Guidance 


The  development  of  project-specific  quality  assurance  plans  is  especially  impor¬ 
tant  for  determining  and  defining  customer  requirements,  industry  practices,  and 
safety  and  environmental  concerns  when  each  project  is  unique  and  particularly 
when  those  requirements  may  conflict.  The  quality  assurance  plans  developed  by 
field  activities  for  each  project  should  define  how  requirements  will  be  met  for 
specific  contracts  or  products. 

The  division  must  plan  and  implement  a  quality  system  that  covers 


♦  preparation  of  a  quality  manual  and  project-specific  quality  management 
plans; 

♦  identification  of  controls,  resources,  and  skills  necessary  to  achieve  the  re¬ 
quirements  specified  in  the  project  management  plan,  quality  assurance 
plans,  or  other  contract; 

♦  update  of  quality  assurance  and  design  review  processes  and  techniques,  as 
necessary; 

♦  compatibility  of  the  design  planning,  design  development,  and  review  and 
approval  procedures;  and 
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♦  identification  and  preparation  of  quality  records  necessary  to  meet  the  re¬ 
quirements  of  the  ISO  9001  elements. 

Guidelines  for  Conformity 

□  Does  the  organization  have  a  quality  manual  that  fully  documents  its  qual¬ 
ity  system,  and  is  that  manual  in  a  form  that  is  readily  accessible  and  un¬ 
derstood  by  all  employees?  Does  the  manual  reference  relevant  standard 
operating  procedures? 

□  Does  the  quality  manual  cover  preparation  of  project  quality  assurance 
plans;  identification  and  acquisition  of  controls,  processes,  inspection  and 
testing  equipment,  design  resources,  and  skills  needed  to  achieve  the  re¬ 
quired  quality;  clarification  of  standards  for  acceptability  of  products; 
compatibility  of  the  design,  design  processes,  and  review  and  approval 
procedures;  and  identification  and  maintenance  of  quality  records? 

□  Has  the  organization  established  and  documented  procedures  for  preparing 
quality  assurance  plans  in  accordance  with  the  specific  customer  require¬ 
ments? 

□  Has  the  organization  established  and  documented  procedures  for  updating, 
as  necessary,  quality  assurance  and  design  review  techniques? 

□  Has  the  organization  established  and  documented  procedures  for  verifying 
compatibility  of  the  design  products  to  its  intended  initieil  requirements — 
in  other  words,  for  ensuring  that  the  customer  gets  what  it  wants? 

4.3  Contract  Review 
4.3.1  General 
Requirement 


This  element  requires  the  division  to  have  a  documented  contract  review  system 
that 

♦  ensures  that  its  customers’  requirements  are  defined,  documented,  and  un¬ 
derstood  before  the  “contract”  is  finalized  (e.g.,  a  properly  developed 
statement  of  work); 

♦  effectively  resolves  misunderstandings  with  customers’  requirements 
when  they  arise; 
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♦  ensures  that  it  is  capable  of  meeting  the  requirements,  with  in-house  per¬ 
sonnel  or  through  subcontracted  support,  before  entering  into  that  agree¬ 
ment  for  engineering  activities;  and 

♦  keeps  records  of  all  above-related  discussions  and  agreements  reached  (see 
4, 16,  Control  of  Quality  Records). 

Note;  “Contract”  refers  to  any  contractual  arrangement— including  MOAs, 

MOUs,  DD  1391s,  delivery  orders,  task  orders,  and  any  other  contracting  vehi¬ 
cle — ^between  the  division  and  its  customer  (internal  or  external). 


Guidance 


If  it  is  to  deliver  a  quality  product,  the  division  must  thoroughly  understand  what 
its  customers  want,  need,  and  expect  before  it  accepts  and  starts  any  work.  And,  it 
is  the  division’s  responsibility  to  ensure  that  those  requirements  and  expectations 
are  effectively  communicated.  The  division  should  not  depend  on  its  customers  to 
express  their  expectations;  instead,  customers  will  expect  the  division  to  know. 


4.3.2  Review 
Requirement 

The  following  are  the  basic  requirements  of  the  contract  review  process: 

♦  Customer  requirements  must  be  adequately  defined,  agreed  to  by  both 
parties,  and  documented  (records  kept). 

♦  Any  final  requirements  that  differ  from  those  in  the  proposal  or  tender 
must  be  resolved. 

♦  The  organization  must  understand  the  requirements  of  the  contract  and 
know  its  capabilities  in  meeting  those  requirements. 


Guidance 


Establishing  and  reviewing  contracts  with  external  customers  is  beyond  the  engi¬ 
neering  or  construction  division’s  authority,  but  making  sure  it  is  capable  of  dehv- 
ering  the  work  promised  is  not.  The  division  should  participate  in  as  many 
preliminary  meetings  as  possible  between  outside  customers  and  the  district’s 
contracting  or  project  management  divisions  to  ensure  that  its  views  are  repre¬ 
sented  and  to  sign  off  on  any  agreements  reached.  Those  contract  review  proce¬ 
dures  should  have  the  following  features; 

♦  An  opportunity  for  all  parties  to  review  the  contract 

♦  A  verification  checkhst 
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♦  A  method  for  questioning  contract  requirements  and  resolving  problems 

♦  Provision  for  changing  the  contract. 

The  organization  should  include  a  finalized  statement  of  work  in  the  quality  as- 
smance  plan  for  each  project  to  serve  as  an  ongoing  reference. 

The  ISO  standards  cover  precontract  tender  arrangements,  as  well  as  contract  and 
ordering  requirements.  The  terms  “contract”  and  “accepted  order”  are  further  de¬ 
fined,  as  any  requirements  agreed  to  by  both  parties,  transmitted  by  any  means 
(including  verbal  orders).  In  that  situation,  the  organization  must  make  sure  those 
requirements  are  agreed  upon  before  their  acceptance. 

4.3.3  Amendment  to  Contract 
Requirement 


All  modifications  to  an  agreed-upon  scope  of  work  must  be  conducted  in  accor¬ 
dance  with  documented  Army,  district,  and  division  procedures;  the  Federal  Ac¬ 
quisition  Regulation  (FAR);  and  FAR  supplements.  As  a  minimum ,  procedures 
must  include  a  review,  notification  to  affected  functions,  and  records  of  the  con¬ 
tract  agreements  reached. 


Guidance 


Only  those  contract  modifications  dealing  with  the  customer’s  changes  to  the  ini¬ 
tially  agreed-upon  requirements  need  to  be  addressed,  other  modifications  are  ad¬ 
dressed  under  4.10.3,  In-Process  Inspection  and  Testing.  Modifications  exceeding 
the  administrative  contracting  officer’s  authority  must  be  coordinated  with  the 
contracting  officer  at  the  district  office  or  above. 

4.3.4  Records 

The  division  must  create  and  maintain  records  of  all  contract  reviews  and  amend¬ 
ments  to  that  contract.  The  personnel  conducting  contract  reviews  and  amending 
contracts  must  be  identified  and  recorded  (see  4.16,  Control  of  Quality  Records). 

Guidelines  for  Conformity 

□  Are  contract  review  activities  applied  to  all  contracts?  Does  the  procedure 
for  contract  review  deal  with  verbal  orders? 

□  Has  the  organization  established,  written,  and  reviewed  its  procedures  to 
ensure  that  all  customer  requirements  are  adequately  defined  and  docu¬ 
mented? 
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□  Has  the  organization  established  written  procedures  for  resolving  conflicts 
with  its  customers  concerning  development  of  the  requirements? 

□  Has  die  organization  established  written  procedures  to  ensure  that  its  con¬ 
tractors  or  in-house  staff  have  the  capability  of  meeting  the  customers’ 
stated  requirements  before  the  contracts  are  signed? 

□  Are  records  of  all  contract  reviews  maintained,  and  are  the  people  who 
conducted  those  reviews  identified? 

4.4  Design  Control 

4.4.1  General 

The  ISO  9001  standards  require  that  the  engineering  division  have  documented 
procedures  for  establishing  project  quality  assurance  plans  for  managing  the  de¬ 
sign  process  of  E&D  products  and  for  verifying  that  those  final  products  meet  the 
requirements  specified  by  the  customers,  internal  policy,  industry  standards,  and 
Army/USACE  regulations. 

Design  control  is  not  applicable  to  quality  system  requirements  of  USAGE  con¬ 
struction  activities  and  is  not  an  element  under  ISO  9002.  However,  on  some 
projects,  changes  to  facility  designs  may  be  necessary.  For  those  cases,  construc¬ 
tion  must  document  how  it  will  handle  such  changes.  The  documentation  should 
include  an  explanation  of  how  the  changes  are  authorized  and  how  they  get  im¬ 
plemented.  Element  4.5.3,  Document  and  Data  Changes,  contains  further  guid¬ 
ance. 

4.4.2  Design  and  Development  Planning 
Requirement 


The  engineering  organization  must  prepare  quality  assurance  plans  for  the  design 
and  development  process  for  every  project;  such  plans  should  reflect  any  prior 
planning  established  in  project  management  plans.  The  plans  must  elaborate  on 
each  engineering  activity  performed  to  ensure  that  the  design  meets  the  cus¬ 
tomer’s  stated  requirements,  industry  specifications,  and  Army/USACE  and  local 
regulations.  The  plan  also  must  assign  the  responsibilities  to  qualified  engineering 
organizations  that  have  adequate  resources  to  perform  the  jobs.  Design  plans  and 
quality  assurance  plans  must  be  approved  by  management  and  must  be  updated,  as 
necessary. 

A  design  planning  framework  using  project-specific  quality  assurance  plans,  will 
♦  show  how  project  designs  are  initiated  and  updated; 
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♦  effectively  demonstrate  who  will  execute  the  design  (in-house  or  an  A-E 
contractor); 

♦  identify  technical  and  informational  interfaces  between  various  engineer¬ 
ing  and  other  district  activities  (e.g.,  show  who  is  in  charge  of  what); 

♦  establish  the  responsibilities  and  the  procedures  for  identifying,  docu¬ 
menting,  verifying,  and  resolving  ambiguous  or  conflicting  design  re¬ 
quirements; 

♦  establish  procedures  for  ensuring  that  completed  designs  conform  to  prop¬ 
erly  developed  customer  requirements  as  well  as  to  any  other  regulatoiy 
requirements; 

♦  assign  design  review  and  approval  responsibilities  (see  4.10,  Inspection 
and  Testing); 

♦  establish  procedures  to  identify  and  document  design  changes  or  modifi¬ 
cations; 

♦  demonstrate  how  the  finished  design  will  meet  the  customer’s  require¬ 
ments;  and 

♦  establish  biddability,  constructability,  and  operability  (BCOE)  review  pro¬ 
cedures  for  construction  division  personnel  to  carry  out. 


Guidance 


The  essential  quality  aspects  of  an  E&D  product,  such  as  cost,  schedule,  environ¬ 
mental  compliance,  safety,  performance,  dependability,  etc.,  are  established  dur¬ 
ing  the  design  planning  phase.  Thus,  deficient  design  planning  can  be  a  major 
cause  of  quality  problems  and,  subsequently,  affect  customer  satisfaction  with  the 
finished  product.  The  engineering  division  should  consult  guidance  established  in 
ER  1 1 10-10-12,  Quality  Management,  for  further  guidance  on  preparing  the  re¬ 
quired  quality  assurance  plans  for  each  project. 

Following  ER  1110-1-12,  project  quality  assurance  plzuis  should  be  sufficiently 
specific  and  detailed  to  permit  effective  verification  during  the  review  and  ap¬ 
proval  steps.  Planning  procedures  should  take  into  account 

♦  sequential  and  parallel  work  activities; 

♦  design  verification  activities; 

♦  safety,  performance,  and  other  quality  aspects  incorporated  in  the  design; 

♦  design  acceptance  criteria;  and 
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♦  assignment  of  responsibilities. 

4.4.3  Organizational  and  Technical  Interfaces 

Design  inputs  and  other  information  used  during  the  design  development  process 
can  come  from  a  variety  of  sources  (internal  engineering  personnel,  the  A-E  con¬ 
tractor,  the  customer,  experts  subcontracted  by  the  customer,  USAGE  headquar¬ 
ters,  etc.).  The  responsibilities  and  authorities  of  those  internal  and  external 
sources  must  be  defined,  documented,  coordinated,  and  controlled.  The  basic  re¬ 
quirement  of  this  element  is  to 

♦  identify  interfaces  (particularly  design  review  and  verification  responsi¬ 
bilities)  between  engineering  division  branches,  between  the  engineering 
division  and  other  district  organizations  (such  as  the  project  management, 
contracting,  construction,  and  resource  management  divisions),  and  be¬ 
tween  the  engineering  division  and  its  customers  and  their  proponents; 

♦  identify  requirements  and  responsibilities  of  the  BCOE  review;  and 

♦  document  and  regularly  review  the  transmission  of  necessary  information. 


4.4.4  Design  Input 
Requirement 


Before  work  begins,  the  engineering  organization  must 

♦  identify  all  design  input  requirements  pertinent  to  the  product  (e.g.,  the 
customer’s  requirements,  engineering  regulations,  local  or  federal  statutes, 
A-E  instructions,  design  specifications,  standard  design  elements,  or  any 
other  design  specifications); 

♦  review  those  requirements  for  adequacy; 

♦  resolve,  with  internal  or  external  customers,  any  incomplete,  ambiguous, 
or  conflicting  requirements;  and 

♦  document  and  maintain  records  of  those  requirements  for  each  project. 


Guidance 


Inputs  for  a  typical  engineering  project  design  can  come  from  a  number  of 
sources.  Specifically,  design  inputs  can  be  design  criteria  and  specifications  from 
the  Corps  of  Engineers  guide  specifications,  other  engineering  division  publica¬ 
tions,  industry  criteria,  and  manufacturers’  specifications.  Other  forms  of  design 
inputs  may  be  user-generated  performance  specifications,  a  statement  of  work, 
A-E  instructions,  and  any  other  applicable  statutory  or  regulatory  requirement. 
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4,4.5  Design  Output 


The  engineering  organization  must  produce  a  finished  E&D  product  (design 
specifications,  drawings,  studies,  etc.)  in  a  format  that  can  be  verified  against  the 
initial  contract  requirements  and  design  inputs  outlined  above.  Engineering  prod¬ 
ucts  must 

♦  meet  the  design  input  requirements, 

♦  contain  or  reference  acceptance  criteria, 

♦  identify  design  characteristics  crucial  to  safety,  and 

♦  be  reviewed  and  approved  by  a  designated  engineering  division  authority 
before  it  is  released  as  a  finished  product. 

4.4.6  Design  Review 

Requirement 


The  engineering  division  must  plan  and  conduct,  at  appropriate  stages,  formal  re¬ 
views  of  the  E&D  product  it  develops.  Dates  and  designated  reviewers  should  be 
specified  in  the  project  management  plan,  the  quality  assurance  plan,  or  both.  Re¬ 
sults  of  those  reviews  must  be  formally  recorded. 


Guidance 


This  element  requires  that  design  reviews  be  performed  but  does  not  specify  when 
in  the  design  development  process  they  must  be  performed  or  who  is  to  perform 
them.  The  current  practice  of  reviewing  the  design  at  the  conceptual,  35  percent, 
60  percent,  90  percent,  and  100  percent  design  completion  stages  is  an  acceptable 
procedure  although  not  necessary  for  every  project  as  long  as  appropriate  reviews 
are  specified  in  the  project  management  and/or  quality  assurance  plan.  In  addition, 
although  not  specifically  mentioned,  BCOE  and  peer  reviews  fall  under  this  ele¬ 
ment.  The  requirement  of  this  ISO  element  merely  states  that,  sometime  during 
the  design  development  planning  stage,  engineering  personnel  should  determine 
the  appropriate  points  for  design  reviews  and  should  identify  who  will  conduct 
them,  their  responsibilities,  and  the  expected  results. 

4.4.7  Design  Verihcation 

Requirement 


The  engineering  organization  must  verify  that  the  in-process  engineering  product 
at  various  stages  of  the  design  process  meets  the  specified  requirements  from 
4.4.4,  Design  Input,  for  that  stage  as  specified  by  the  project  management  plan  or 
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quality  assurance  plan.  Records  of  design  verification  and  the  measures  used  must 
be  recorded  (see  4.16,  Control  of  Quality  Records). 


Guidance 

This  element  addresses  the  logical  outcome  from  performing  design  reviews, 
which  is  to  verify  that  the  design  meets  the  appropriate  design  input  requirements 
at  each  stage.  Design  verification  involves  the  individuals  that  perform  design  re¬ 
views.  Examples  of  design  verification  include  the  following: 

♦  Customary  engineering  design  reviews — ^use  of  the  Automated  Review 
Management  System  (ARMS)  is  recommended 

♦  BCOE  and  peer  reviews 

♦  Comparisons  of  new  design  elements  to  similar  proven  elements  of  previ¬ 
ous  designs  or  to  design  standards  or  Corps  guide  specifications 

♦  Customer  review  and  approval  to  proceed  with  the  next  stage  design 

♦  Tests  of  the  mockup  of  the  completed  E&D  product  (or  sections  of  the 
product)  to  ensure  that  the  full-scale  version  will  work  as  planned 

♦  Tests,  computer  simulations,  or  demonstrations  during  the  design  stage. 

4.4.8  DESIGN  VALIDATION 

Design  validation  must  be  performed  on  all  E&D  products  to  ensure  that  the 
product  sufficiently  satisfies  the  customers’  intended  requirements  as  specified  in 
the  contract.  The  engineering  division  should  validate  its  product  through  the 
customer  and  construction  division  prior  to  project  completion. 

Note:  Sometimes,  validation  of  the  design  will  be  possible  only  after  the  project  is 
actually  constructed.  Since  construction  is  not  the  responsibility  of  the  engineering 
organization  (and  may  not  be  applicable  to  all  projects),  the  engineering  organiza¬ 
tion  can  meet  the  requirements  of  this  element  through  performance  of  design 
criteria  feedback  inspections. 

4.4.9  Design  Changes 
Requirement 

All  changes  to  approved  designs  must  be  identified,  documented,  reviewed,  and 
approved  by  authorized  engineering  division  personnel  before  those  changes  are 
implemented. 


A-21 


Guidance 


This  element  applies  both  to  amendments  made  during  the  design  development 
process  and  to  design  modifications  after  construction  contract  award.  Sometimes, 
improving  one  characteristic  may  have  an  unforeseen  adverse  influence  on  an¬ 
other.  The  changes  to  the  design  should  be  communicated  to  all  concerned 
(especially  the  customer)  and  should  be  documented. 

Guidelines  for  Conformity 

Q  Do  the  quality  assurance  plans  identify  the  responsibility  of  each  organi¬ 
zation  involved  in  the  project,  specify  reasonable  schedules,  identify  major 
activities,  and  review  points,  etc.? 

Q  Are  the  quality  assurance  plans  updated  as  the  project  evolves? 

Q  Is  the  project  schedule  adequately  developed,  and  are  the  various  project 
activities  assigned  to  qualified  personnel  equipped  with  resources  to  com¬ 
plete  it  effectively? 

Q  Are  orgamzation  and  technical  interfaces  between  different  engineering 
organizations  and  other  district  elements  identified,  documented,  and  re¬ 
viewed  regularly? 

Q  Are  the  organization’s  personnel  notified  of  changes  as  the  project 
evolves? 

□  Are  customer  requirements  defined  and  incorporated  during  the  design  de¬ 
velopment  process?  Are  adequate  control  mechanisms  in  place  to  ensure 
that  they  are  met? 

□  Are  statutory/regulatory  requirements  addressed  during  project  planning? 

Q  Are  procedures  in  place  for  resolving  incomplete,  ambiguous,  or  conflict¬ 
ing  requirements  during  the  design  development  process? 

□  Are  design  verifications  planned  and  conducted? 

□  Are  various  design  elements  validated? 

□  Are  the  quality  assurance  plans  checked  to  ensure  that  they  do  not  conflict 
with  the  customers’  initial  requirements  addressed  in  the  contract  and 
project  management  plans? 

Q  Are  procedures  adequate  for  identifying,  documenting,  reviewing,  and  ap¬ 
proving  all  potential  changes  and  modifications  to  the  projects? 
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□  Are  procedures  in  place  for  conducting  BCOE  and  peer  reviews,  and  are 
those  reviews  effectively  carried  out  for  each  project  when  appropriate? 

□  Is  the  ARMS  effectively  used? 

4.5  Document  and  Data  Control 

4.5.1  General 

Requirement 


The  engineering  or  construction  division  must  have  documented  procedures  for 
creating,  distributing,  controlling,  reviewing,  approving,  and  publishing  internally 
and  externally  generated  documents  and  data  that  relate  to  any  applicable  element 
of  this  standard  or  for  any  project.  The  organization  will  identify  the  documents 
and  data  it  will  control. 

Note:  Documents  and  data  can  be  of  any  medium — ^hard  copy  or  electronic. 


Guidance 


Document  control  applies  to  aU  documents  and  data  pertinent  to  project  planning, 
purchasing,  supervision  and  review,  quality  standards,  design  reviews  and  verifi¬ 
cation,  testing,  and  internal  written  procedures.  Internal  written  quality  system 
procedures  covering  this  element  should  describe 

♦  what  documents  and  data  are  to  be  controlled, 

♦  how  documentation  for  those  functions  should  be  controlled, 

♦  who  is  responsible  for  document  control  (may  be  more  flian  one  person 
and  specific  documents  may  be  controlled  by  different  people),  and 

♦  where  and  when  that  control  is  to  occur. 

The  following  are  examples  of  documents  and  data  that  the  division  may  wish  to 
control: 

♦  Applicable  federal  statutes 

♦  All  Army  and  USAGE  regulations,  pamphlets,  circulars,  and  technical 
notes 

♦  Corps  of  Engineers  guide  specifications 

♦  Industry  and  local  design  standards  and  relevant  codes 
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♦  Standard  designs 

♦  Project  specifics  (e.g.,  construction  contract  plans  and  specifications) 

♦  ISO  9000  quality  manual  and  quality  system  procedures 

♦  Relevant  work  instructions 

♦  Training  materials 

♦  Any  other  ISO-related  quality  documentation. 

4.5.2  Document  and  Data  Approval  and  Issue 
Requirement 


The  division  must  prepare  a  master  list  of  documents  and  data  that  it  intends  to 
control.  The  list  should  include  documents  and  data  required  by  the  customer. 
Authorized  personnel  must  approve  each  entry  on  the  master  list  before  it  is  is¬ 
sued. 


Guidance 


To  avoid  using  invalid  or  obsolete  documents,  division  personnel  must  be  able  to 
identify  the  current  revision  status  of  the  controlled  documents  and  data,  and  they 
must  ensure  that  current  and  appropriate  documents  and  data  are  available  at  all 
relevant  locations  where  needed.  The  division  must  remove  invalid  or  obsolete 
controlled  documents  or  otherwise  handle  them  to  ensure  against  unintended  use, 
and  it  must  identify  any  obsolete  documents  retained  for  legal  or  knowledge- 
preservation  purposes. 

4.5.3  Document  and  Data  Changes 

Requirement 


When  changes  to  controlled  documents  or  data  are  necessary,  cognizant  engi¬ 
neering  or  construction  activities  or  personnel  must  identify  and  approve  the 
changes.  The  review  and  approval  process  must  be  performed  by  the  same  activi¬ 
ties  or  people  that  performed  the  original  review  (unless  specifically  designated 
otherwise);  those  activities  or  people  must  have  access  to  appropriate  information 
upon  which  to  base  their  decisions.  The  organization  must  consider  whether  the 
changes  should  be  identified,  either  within  the  documents  that  have  been  changed 
or  in  appropriate  change  notifications. 
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Guidance 


This  element  applies  to  all  internally  and  externally  controlled  documents  and 
data.  The  division  should  consider  the  effect  that  changes  in  one  area  may  have  on 
other  parts  of  the  organization  and  should  plan  the  circulation  of  a  change  pro¬ 
posal  to  avoid  disruption  as  well  as  time  the  implementation  to  minimize  disrup¬ 
tion. 

Guidelines  for  Conformity 

□  Does  a  master  list  of  all  essential  documentation  exist  to  ensure  that  the 
most  up-to-date  documents  are  in  place  where  they  are  needed  and  to  en¬ 
sure  that  outdated  or  nonapplicable  documents  are  never  used? 

□  Are  documents  approved  before  being  issued? 

□  Are  mechanisms  in  place  for  controlling  the  issue  of  all  pertinent  docu¬ 
ments? 

□  Are  document  changes  reviewed  and  approved  by  the  same  activities  that 
performed  the  original  issuance  (if  issued  internally)? 

□  Before  being  entered  into  the  system,  are  all  documents  checked  for  ap¬ 
propriate  levels  of  review  and  approval? 

□  Is  there  a  policy  for  reissuing  documents  after  a  certain  number  of  revi¬ 
sions/changes? 

4.6  Purchasing 
4.6.1  General 
Requirement 


This  element  requires  that  the  division  have  documented  procedures  to  ensure  that 
all  purchased  goods  and  services  conform  to  the  requirements  set  forth  in  the 
contract,  project  management  plan,  and/or  quality  assurance  plan.  In  addition,  this 
element  requires  that  all  A-E,  construction,  and  other  service  contractors  he  se¬ 
lected  on  the  basis  of  their  ability  to  meet  the  requirements  specified  in  the  con¬ 
tract  (see  4.3,  Contract  Review).  The  acquisition  process  employed  for  all 
engineering  or  construction  subcontractors  must  conform  to  the  requirements  of 
local  contracting  procedures,  the  FAR,  and  FAR  supplements. 

Note:  This  element  does  not  apply  to  engineering  or  construction  organizations 
that  do  not  need  to  purchase  goods  or  services  in  order  to  deliver  their  product  or 
that  have  little  or  no  influence  over  the  selection  of  the  general  contractors. 
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Guidance 


Purchasing  procedures  that  are  planned  and  adequately  controlled  ensure  that  sub¬ 
contracted  products  conform  to  the  specified  requirements.  The  division  should 
establish  effective  working  relationships  and  feedback  systems  with  all  its  sub¬ 
contractors,  engineering  laboratories,  general  construction  contractors,  and  others. 
A  quality  procurement  program  should  include 

♦  quality  assurance  procedures  for  planning  and  controlling  subcontracted 
products, 

♦  criteria  for  selecting  acceptable  subcontractors, 

♦  agreement  with  subcontractors  on  quality  assurance, 

♦  agreement  with  subcontractors  on  verification  methods, 

♦  provisions  for  settling  disputes  between  USAGE  and  the  subcontractor, 
and 

♦  quality  records  related  to  purchasing. 

4.6.2  Evaluation  of  Subcontractors 
Requirement 


By  law,  procurement  of  A-E  and  construction  contractors  is  the  responsibility  of 
the  district’s  contracting  officer  and  must  comply  with  the  FAR  and  FAR  supple¬ 
ments.  However,  when  requested,  the  engineering  or  construction  division  must 
evaluate  those  contractors  and  their  abilities  to  make  sure  that  they  are  capable  of 
fulfilling  the  project  requirements. 

During  execution  of  any  contract,  division  personnel  must  monitor  the  contrac¬ 
tor’ s  performance  against  the  contract  and  provide  the  necessary  feedback  to  the 
contracting  authorities  so  that  poor  performers  can  be  ehminated  from  considera¬ 
tion  for  subsequent  contracts.  The  organization  must 

♦  evaluate  and  select  contractors  based  on  their  ability  to  meet  the  require¬ 
ments, 

♦  establish  and  maintain  records  of  acceptable  and  unacceptable  contractors, 
and 

♦  define  the  type  and  extent  of  control  exercised  over  subcontractors. 
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Guidance 


Divisions  that  properly  use  the  A-E  Contract  Administration  Support  System 
(ACASS)  and  the  Construction  Contract  Administration  Support  System 
(CCASS)  will  meet  the  requirements  of  this  element.  The  subcontractor’s  quality 
records  should  be  sufficiently  comprehensive  to  demonstrate  its  ability  to  meet  the 
project  requirements.  The  division  may  employ  one  of  several  methods  for 
choosing  satisfactory  contractors.  For  example,  a  subcontractor  may  be  assessed 
on  the  basis  of  previous  performance,  registration  to  ISO  9000  or  other  quality 
system  standard,  or  some  other  appropriate  quality  system  standard. 

4.6.3  PURCHASING  Data 

For  all  services,  materials,  and  supplies  purchased  by  the  engineering  or  construc¬ 
tion  organization,  the  required  contracting  documents  must  clearly  identify  what  is 
being  purchased.  Where  applicable,  the  division  must  support  the  contracting  offi¬ 
cer  in  developing  the  statements  of  work  for  products  or  services  needed.  The  re¬ 
quirements  for  purchasing  data  include  the  following: 

♦  In  the  purchasing  document,  clearly  and  specifically  describe  the  product 
or  service  required. 

♦  Review  and  approve  purchasing  documents  for  adequacy  of  specified  re¬ 
quirements. 

4.6.4  Verification  of  Purchased  Product 

Conformity  of  the  subcontracted  product  to  specifications  may  be  verified  by  (1) 
USAGE  engineering  or  construction  staff  at  the  contractor’s  or  subcontractor’s 
premises  or  (2)  the  customer  at  the  contractor’s  or  subcontractor’s  premises, 
USAGE  premises,  or  the  point  of  installation.  In  the  first  situation,  USAGE  must 
“specify  verification  arrangements  and  the  method  of  product  release  in  the  pur¬ 
chasing  documents.”  In  the  second  situation,  the  ISO  standard  adds  two  caveats: 

♦  Verification  by  the  customer  cannot  be  used  by  the  engineering  or  con¬ 
struction  organization  as  evidence  of  effective  quality  control  by  the  sub¬ 
contractor. 

♦  Verification  by  the  customer  does  not  absolve  USAGE  of  responsibility 
for  providing  a  quality  product. 

Guidelines  for  Conformity 

□  Do  procedures  and  specifications  exist  for  all  goods  and  services  pur¬ 
chased  or  contracted  by  the  organization?  Is  there  a  system  in  place  to  en¬ 
sure  that  all  purchased  goods  and  services  conform  to  those  specifications? 
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□  Are  contractors  selected  on  the  basis  of  their  ability  to  meet  requirements? 

□  Does  the  organization  currently  evaluate  the  quality  systems  used  by  con¬ 
tractors,  and  are  the  quality  systems  evaluated  on  the  basis  of  past  experi¬ 
ence,  on-site  visits,  and/or  their  registration  under  the  ISO  or  similar 
standards? 

□  Are  contracts,  purchase  orders,  and  delivery  orders  always  reviewed  to  en¬ 
sure  that  they  contain  the  necessary  requirements? 

□  Is  ACASS  routinely  queried  before  subcontracting  for  A-E  services? 

4.7  Control  of  Customer-Supplied  Product 

General 

Requirement 

The  engineering  or  construction  division  must  document  its  procedures  for  en¬ 
suring  that  any  materials  or  products  supplied  by  the  customer  for  use  in  the  prod¬ 
uct  to  be  delivered  to  the  customer  are  properly  stored  and  maintained  and  are 
verified  before  they  are  used.  The  loss  of  or  damage  to  any  customer-supplied 
products,  as  well  as  the  delivery  of  unsuitable  products  supplied  by  the  customer, 
must  be  recorded  and  reported  to  the  customer.  The  standards  require  that  records 
of  all  customer-supplied  products  be  kept. 

Guidance 


Most  USAGE  engineering  divisions  rarely  receive  tangible  materials  from  their 
customers  for  inclusion  in  an  E&D  product.  In  nearly  every  situation,  the  materi¬ 
als  supplied  by  their  customers  are  documentation  or  data  and  therefore  fall  under 
the  provisions  of  4.5,  Document  and  Data  Control,  to  satisfy  the  requirement  for 
proper  handling  and  verification  before  use;  the  provisions  of  4.16,  Control  of 
Quality  Records,  to  satisfy  the  requirements  for  proper  storage  and  maintenance; 
and  the  provisions  of  4.13,  Control  of  Nonconforming  Product,  to  satisfy  the  re¬ 
quirements  of  reporting  unsuitable  conditions  back  to  the  customers. 

Documentation  and  data  that  may  be  received  from  the  customer  and  that  faU  un¬ 
der  those  requirements  may  include  local  engineering  or  design  specifications, 
legal  documentation,  design  criteria,  product  brochures,  computer  equipment, 
software,  as-built  drawings,  local  design  standards,  or  any  other  items  owned  by 
USAGE  customers  and  furnished  for  use  in  meeting  the  contract  requirements. 

The  engineering  division  may  wish  to  develop  documented  procedures  covering 
the  requirements  of  this  element  in  the  event  it  ever  does  take  possession  of  mate¬ 
rials  other  than  documentation  and  data. 
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Construction  divisions  may  receive  government-furnished  materials  procured  by 
the  customer  for  use  in  a  constructed  facility.  They  may  also  be  any  materials  pro¬ 
cured  by  the  district  contracting  activity  to  use  as  the  construction  site,  finally, 
customer-supplied  products  may  be  specifications,  legal  documentation,  product 
brochures,  blueprints,  compute  equipment,  software,  procedure  manuals,  or  any 
other  items  that  are  owned  by  the  customer  and  furnished  to  the  construction  or¬ 
ganization  for  use  in  meeting  the  contract  requirements. 

Guidelines  for  Conformity 

□  Are  customers’  as-built  drawings,  design  guides,  procedure  and  operating 
manuals,  and  preUminary  design  documents  effectively  controlled  ac¬ 
cording  to  4.5,  Document  and  Data  Control,  while  diey  are  being  used,  and 
if  requested,  are  they  returned  to  the  customers  when  the  project  is  com¬ 
plete? 

4.8  Product  Identification  and  Traceability 

General 

Requirement 

The  division  must  have  a  documented  system  to  ensure  that  its  products  are 
uniquely  identifiable  and  traceable  through  all  stages  of  the  project.  For  example, 
if  errors  in  the  end  product  are  discovered,  the  designs  must  be  traceable  to  the 
stage  at  which  problems  occurred  so  that  the  organization  can  perform  an  effec¬ 
tive  audit  of  the  root  causes  of  the  problems. 

Guidance 

Many  identification  methods  exist,  but  at  a  minimum,  all  products  (plans  and 
specifications,  drawings,  studies,  etc.)  should  be  numbered  according  to  a  prees¬ 
tablished  convention.  The  identifier  should  be  unique  to  the  project  and  the  source 
of  the  operation.  Separate  identifiers  could  be  required  to  record  changes  in  vari¬ 
ous  aspects  of  the  project.  Traceability  may  require  that  specific  personnel  in¬ 
volved  in  phases  of  the  operation  be  identified,  which  can  be  accomplished,  for 
example,  through  signatures  on  serially  numbered  documents. 

Guidelines  for  Conformity 

□  Are  all  designs  and  products  adequately  and  uniquely  identified? 

□  Is  a  system  in  place  to  ensure  that  products  are  traceable? 


A-29 


4.9  Process  Control 

General 

Requirement 


The  organization  must  plan,  identify,  and  document  the  processes  and  activities 
that  directly  affect  the  quality  of  the  finished  product.  In  addition,  those  proc¬ 
esses/activities  must  be  carried  out  in  a  controlled  maimer.  Consideration  must  be 
given  to  the  following: 

♦  Documenting  procedures  and  work  instructions,  using  suitable  test  equip¬ 
ment  or  computer  models,  and  ensuring  suitable  working  environments 

♦  Knowing  and  complying  with  applicable  federal  and  state  laws,  environ¬ 
mental  regulations,  industry  codes,  design  criteria,  safety  standards.  Army 
and  USAGE  regulations,  etc. 

♦  Developing  criteria  for  determining  acceptable  performance 

♦  Monitoring  the  progress  of  projects  against  the  project  management  and 
quality  plans  and  resolve  problems  that  arise  (design  errors,  differing  site 
conditions  etc.). 


Guidance 


Preventing  problems  by  controlling  processes  is  better  than  discovering  problems 
after  the  project  has  been  completed.  To  get  its  processes  under  control,  a  division 
should  consider 

♦  conducting  process  capability  studies  to  evaluate  the  effectiveness  of  its 
processes, 

♦  developing  work  instructions  that  describe  the  criteria  for  determining 
satisfactory  work  completion  and  conformity  to  standards  of  good  work¬ 
manship,  and 

♦  verifying  the  quality  status  of  the  product  and  process. 

Documented  work  instructions  should  include 

♦  methods  of  accomplishing  the  work, 

♦  tools  and  equipment  needed  to  perform  the  tasks, 

♦  sequences  of  activities. 
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♦  monitoring  requirements, 

♦  necessary  review  and  approval  requirements, 

♦  standard  designs  used  when  and  where  appropriate,  and 

♦  sampling  requirements,  where  appropriate. 

For  USAGE  engineering  organizations,  this  element  identifies  the  same  quality 
system  requirement  as  4.4,  Design  Control,  since  design  development  is  the  engi¬ 
neering  division’s  primary  activity.  Therefore,  requirements  of  4.4,  Design  Con¬ 
trol,  and  4.9,  Process  Control,  should  be  considered  complementary.  However, 
this  element  adds  the  requirement  for  control  of  equipment  used  and  service  ac¬ 
tivities. 

Guidelines  for  Conformity 

□  Have  adequate  procedures  and  work  instructions — ^for  example,  critical 
path  activity  networks — ^been  estabhshed  describing  how  to  perform  all 
activities? 

□  Are  critical  activities  monitored? 

□  Do  standards  for  what  constitutes  acceptable  performance  exist  in  relation 
to  performance  of  process  activities? 

□  Do  personnel  know  what  variables  in  the  process  are  important  to  achieve 
excellent  quality? 

□  Are  detailed  process  instructions  and  procedures  established  and  well 
documented,  and  are  they  easily  understood  by  those  who  must  use  them? 

4.10  Inspection  and  Testing 

4.10.1  General 

Requirement 


The  division  must  have  documented  systems  and  procedures  for  effectively  in¬ 
specting  and  testing  (e.g.,  reviewing  and  approving)  all  incoming  products  from 
outside  subcontractors  and  vendors  (e.g.,  A-E  contractors),  all  in-process  activi¬ 
ties,  and  the  final  product  before  it  is  delivered  to  the  customers  to  ensure  that  the 
product  conforms  to  the  requirements  established  in  the  project  management  plan 
and  quality  assurance  plan. 
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Guidance 


Since  the  primary  responsibility  of  an  engineering  division  is  design  development, 
the  design  review  and  verification  provisions  required  by  4.4,  Design  Control,  are 
essentially  the  same  provisions  directed  by  4.10,  Inspection  and  Testing.  There¬ 
fore,  the  two  separate  quality  system  requirements  should  be  considered  comple¬ 
mentary. 

4.10.2  Receiving  Inspection  and  Testing 
Requirement 


When  it  receives  a  product  from  any  of  its  contractors  (see  4.6,  Purchasing),  the 
division  must  ensure  that  the  product  fulfills  contractual  obhgations  and  that  no 
subcontracted  work  is  used  or  further  processed  until  it  has  been  verified  and  ap¬ 
proved  in  accordance  with  the  contract  requirement,  project  management  plan, 
quality  assurance  plan,  and  other  documented  procedures.  However,  for  any  work 
required  for  urgent  release,  the  division  must  ensure  that  the  product  is  properly 
and  uniquely  identified  to  permit  positive  recall,  if  necessary.  A  log  of  il  incom¬ 
ing  and  rejected  products  must  be  maintained. 


Guidance 


Nearly  all  of  an  engineering  division’s  incoming  products  will  be  from  A-E  con¬ 
tractors,  test  laboratories,  or  engineering-related  inspections;  therefore,  4.6,  Pur¬ 
chasing,  is  complementary  to  this  element  of  the  standard.  This  element  does  not 
imply  that  incoming  items  must  be  inspected  and  tested  if  the  engineering  organi¬ 
zation  is  confident  that  other  quality  verification  procedures  would  suffice.  Qual¬ 
ity  verification  procedures  may  include 

♦  provisions  for  verifying  contracted  A-E  work,  materials,  or  engineering 
services  against  preestablished  requirements,  and 

♦  provisions  for  taking  action  when  subcontracted  work  does  not  conform. 

The  engineering  organization  should  keep  appropriate  records  to  ensure  the  avail- 
abihty  of  historical  data  to  assess  A-E  subcontractor  performance  and  quality 
trends.  The  records  should  serve  as  input  into  ACASS. 

Note;  The  urgent  release  of  any  E&D  products,  even  if  subject  to  recall,  should 
generally  be  discouraged  as  a  matter  of  good  quaUty  management  practice. 

For  a  construction  division  project,  personnel  must  verify  that  off-site  testing  is 
being  performed  in  accordance  with  the  quahty  assurance  plan  or  with  the 
contractor’s  quality  control  plan  and  industry  guidelines  and  must  ensure  that  the 
test  samples  apply  to  the  product  to  be  supplied.  For  building  equipment  or 
materials  inspected  off  site,  verification  must  be  performed  per  4.6.4,  Verification 
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of  Purchased  Product.  When  the  equipment  or  material  is  delivered,  construction 
must  vahdate  that  it  is  the  same  equipment  or  material  examined  and  approved  off 
site.  When  construction  uses  materials  before  they  are  validated,  it  must  document 
where  they  were  used.  For  example,  if  it  pours  concrete  before  it  receives 
compression  test  results,  construction  should  document  the  location  of  that  pour. 

4.10.3  In-Process  Inspection  and  Testing 

Requirement 


The  division  must  ensure  that  in-process  testing  and  inspection  are  performed  as 
required  by  the  project  management  plan,  quality  assurance  plan,  or  the  contrac¬ 
tor’s  quality  control  plan.  It  should  not  permit  the  work  to  advance  until  the  re¬ 
quired  testing  and  inspections  have  been  completed  and  approved.  (For  example, 
local  inspections  of  electrical  work  may  be  necessary  before  walls  can  be  closed.) 
The  exception  is  when  work  is  released  under  positive  recall  procedures;  the  re¬ 
lease  under  positive  recall  procedures,  however,  would  not  preclude  the  test  and 
inspection  steps  required  above. 

Nonconforming  work  should  be  handled  according  to  4.13,  Control  of  Noncon¬ 
forming  Product,  except  when  work  is  released  under  positive  recall  procedures. 
The  release  under  positive  recall  procedures,  however,  would  not  preclude  the 
previous  review  and  approval  steps  required  above.  Contract  modifications 
needed  because  of  design  errors  or  differing  site  conditions  must  be  handled  ac¬ 
cording  to  the  contract  requirements,  FAR,  and  FAR  supplements. 


Guidance 


In-process  inspection  and  testing  applies  to  all  forms  of  products.  It  allows  non¬ 
conformities  to  be  found  early  and  resolved.  Statistical  control  techniques  can  be 
used  to  identify  process  trends  that  are  out  of  control  and  to  prevent  future  prob¬ 
lems  (see  4.20,  Statistical  Techniques). 

4.10.4  Final  Inspection  and  Testing 

The  division,  in  accordance  with  documented  procedures  and  with  the  project 
management  and  quality  assurance  plans,  must  carry  out  final  test  and  inspection 
of  the  product  before  it  is  delivered  to  the  customer.  The  customer  also  may  con¬ 
duct  a  final  inspection.  The  final  review  and  acceptance  criteria  must  be  specified 
in  the  quality  management  plan  or  other  documented  procedures. 

4. 10.5  Inspection  and  Test  Records 

Requirement 


The  division  must  establish  and  maintain  records  that  indicate  whether  the  product 
has  passed  in-process  and  final  inspections  and  tests  done  in  accordance  with  the 
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project  management  plan  and  quality  assurance  plan.  Those  records  must  identify 
the  inspection  audiority  responsible  for  releasing  the  product.  The  records  may 
include  punch  lists,  local  buildings  and  trade  inspections,  building  certification 
forms,  transfer  deeds,  and  inspection  logs. 


Guidance 


Records  must  be  kept  in  accordance  with  4.16,  Control  of  Quality  Records.  They 
will  facilitate  future  assessments,  including  regulatory  compliance  and  possible 
liability  issues. 

Guidelines  for  Conformity 

□  Do  documented  procedures  exist  that  identify  what  review  and  validation 
is  required  and  what  records  must  be  established? 

□  Is  there  a  system  in  place  to  ensure  that  the  work  contracted  to  outside 
firms  and  other  purchased  services  is  of  acceptable  quality? 

□  Are  inspection  and  review  procedures  performed  in  accordance  with  the 
quality  assurance  plan,  project  management  plan,  or  other  documented 
procedures? 

□  Are  conforming  and  nonconforming  products  clearly  identified,  and  does 
the  system  provide  procedures  for  rejecting  unacceptable  work? 

□  Do  product  failures  trigger  procedures  for  centrally  controlling  noncon¬ 
forming  products? 

□  Are  procedures  in  place  for  reviewing  and  approving  in-process  designs, 
and  do  those  procedures  handle  the  rejection  of  in-process  work? 

□  Are  all  designs  released  only  after  a  final  review  confirms  that  release  of 
the  designs  and  other  products  is  acceptable? 

□  Are  records  maintained  showing  that  designs  have  passed  a  final  review 
and  specifying  what  the  final  review  criteria  were  and  who  authorized  their 
release? 
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4.1 1  Control  of  Inspection,  Measuring,  and  Test  Equipment 

4.11.1  General 

The  engineering  or  construction  organization  that  uses  in-house  or  field  inspection 
and  testing  equipment  must  have  documented  procedures  for  controlling  that 
equipment.  The  organization  should 

♦  identify  which  measuring  and  test  equipment  must  be  controlled  and  cali¬ 
brated; 

♦  establish  and  maintain  documented  procedures  to  control,  calibrate,  and 
nnaintain  any  equipment  used  for  reviewing,  validating,  measuring,  and 
testing  that  demonstrate  conformance  to  established  requirements; 

♦  use  any  such  equipment  in  a  manner  that  ensures  that  measurement  uncer¬ 
tainty  is  known  and  is  consistent  with  the  required  measurement  capabil¬ 
ity; 

♦  check  and  recheck  the  capability  of  any  test  software  or  hardware  used  as 
forms  of  design  review,  cost  estimating,  or  engineering  analysis;  and 

♦  when  requested  by  file  customer,  provide  technical  data  for  any  needed 
equipment. 

4. 1 1 .2  Control  Procedure 

Requirement 


Where  required,  the  organization  must  identify  the  necessary  measurements;  the 
accuracy  required;  the  appropriate  inspection,  measuring,  and  test  equipment 
needed;  and  the  computer  or  test  models  to  be  used  during  the  review  and  verifi¬ 
cation  process.  Where  such  equipment  is  deemed  necessary,  the  organization  must 

♦  identify,  calibrate,  and  adjust  all  equipment  in  accordance  with  the  quality 
assurance  plan,  relevant  American  Society  for  Testing  and  Materials  stan¬ 
dards,  relevant  military  standards,  and  test  equipment  manufacturers’  in¬ 
structions; 

♦  establish,  document,  and  maintain  calibration  procedures; 

♦  ensure  that  the  equipment  used  is  capable  of  the  required  accuracy  and 
precision; 

♦  be  able  to  identify  equipment  to  indicate  calibration  status; 
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♦  assess  the  validity  of  previous  results  when  equipment  is  out  of  calibra¬ 
tion; 

♦  ensure  suitable  conditions  for  calibration,  inspection,  measurement,  and 
testing; 

♦  ensure  accuracy  and  fitness  for  use  when  handUng,  preserving,  and  storing 
equipment; 

♦  safeguard  inspection,  measuring,  and  test  facilities;  and 

♦  mainfcun  calibration  records. 

Guidance 

The  control  of  measuring  and  test  equipment  and  test  methods  should  address  the 
following  factors,  as  appropriate: 

♦  Suitable  specification  and  acquisition 

♦  Initial  calibration  prior  to  first  use  to  validate  required  bias  and  precision 

♦  Periodic  recall  for  adjustment  and  recalibration  to  maintain  required  accu¬ 
racy  in  use 

♦  Documentary  evidence  that  covers  instrument  identification,  calibration 
status,  and  handling  procedures 

♦  Traceability  to  accurate  and  stable  reference  standards. 

Guidelines  for  Conformity 

□  Is  test  equipment  (e.g.,  electronic  survey,  piezometers,  and  survey  boat 
equipment)  properly  controlled  and  periodically  calibrated? 

□  Are  computer  models  periodically  reviewed  and  updated  as  the  model  as¬ 
sumptions,  boundaries,  and  analytic  data  change? 

□  Does  the  organization  maintain  records  of  calibrations — ^the  results,  accu¬ 
racy,  and  precision — and  the  status  of  the  equipment? 
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4.12  Inspection  and  Test  Status 

General 

Requirement 


The  engineering  or  construction  division  must  ensure  that  the  inspection  and  test 
status  of  each  product  is  identified.  Status  identification  ensures  that  the  product 
being  designed  is  acceptable.  In  addition,  the  people  responsible  for  releasing  fin¬ 
ished  products  must  be  identified  and  documented. 


Guidance 


The  status  identifier  should  be  part  of  the  same  preestablished  convention  defined 
in  4.8,  Product  Identification  and  Traceability,  and  should  indicate  whether  the 
product 

♦  has  not  been  reviewed, 

♦  has  been  reviewed  and  accepted  to  what  design  stage, 

♦  has  been  reviewed  and  is  on  hold  awaiting  decision,  or 

♦  has  been  reviewed  and  rejected  (see  4.13,  Control  of  Nonconforming 
Product). 

The  use  of  the  ARMS  is  appropriate  and  satisfies  the  intent  of  this  element,  but 
verification  status  could  be  identified  by  any  suitable  means  such  as  stamping,  de¬ 
sign  approvals,  approval  blocks  on  reports  or  studies,  notations,  review  records 
that  accompany  the  product,  or  by  physical  location. 

Guidelines  for  Conformity 


□  Is  both  conforming  and  nonconforming  design  work  marked  to  indicate 
current  status,  and  do  other  engineering  or  construction  personnel  have  ac¬ 
cess  to  and  know  the  status  of  in-process  work? 

□  Do  those  personnel  who  utilize  contracted  work  know  what  its  quality  is  or 
its  level  of  acceptability? 

□  Do  personnel  know  who  is  able  to  release  work  or  reject  unacceptable 
work? 

□  Is  a  system  in  place  for  releasing  acceptable  products? 

□  Are  adequate  records  kept  to  indicate  the  status  and  inspection  authority 
responsible  for  releasing  finished  designs? 
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4.13  Contool  of  Nonconforming  Product 

4.13.1  General 
Requirement 

The  inadvertent  use  of  any  nonconforming  product  that  does  not  meet  stated  re¬ 
quirements  must  be  minimized.  The  organization  must  have  systems  and  proce¬ 
dures  in  place  to  ensure  that  no  product  wifli  identified  errors  reaches  the 
customer.  All  reworked  products  must  be  reinspected  in  accordance  with  the  pro¬ 
visions  of  the  quality  assurance  plan. 

Guidance 

A  nonconforming  product  is  one  fliat  fails  to  meet  the  stated  requirements  or  in¬ 
dustry  specifications.  Controlling  a  nonconforming  product  requires  procedures 
for 

♦  identifying  and  documenting  nonconforming  components  of  a  product, 

♦  evaluating  the  nonconformity, 

♦  considering  alternatives  for  correcting  the  nonconforming  products, 

♦  physically  controlling  the  further  processing  of  the  nonconforming  prod¬ 
ucts,  and 

♦  notifying  all  activities  that  may  be  affected  by  the  nonconformity. 

4.13.2  Review  and  Disposition  of  Nonconforming  Products 
Requirement 


The  division  must  define  who  is  responsible  for  reviewing  and  authorizing  the 
disposal  of  nonconforming  products  and  must  document  what  is  to  be  done  with 
deficient  product.  Nonconforming  products  may  be  reworked,  accepted  without 
being  reworked  by  concession  of  the  customer,  or  rejected. 


Guidance 


The  division  should  that  actions  to  deal  with  nonconforming  products  as  soon  as 
indications  occur  that  materials,  components,  or  the  completed  product  do  not  or 
may  not  meet  the  specified  contractual  requirements.  The  people  who  review 
nonconforming  items  should  be  competent  to  evaluate  the  effects  of  the  decision 
of  interchangeability,  further  processing,  performance,  reliability,  safety,  and 
aesthetics.  A  decision  to  accept  a  nonconforming  product  should  be  documented. 
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together  with  the  reason  for  doing  so,  in  authorized  waivers,  with  appropriate 
precautions. 

Guidelines  for  Conformity 

□  Does  the  organization  have  procedures  for  handling  unacceptable  work? 

□  Does  the  organization  identify  unacceptable  work  and  ensure  that  unac¬ 
ceptable  work  is  never  released  to  its  customers? 

□  Are  adequate  records  kept  indicating  the  disposition  of  unacceptable 
work? 

□  Is  a  procedure  in  place  for  reworking  unacceptable  work  and  making  it 
comply  with  the  appropriate  standards  and/or  the  customer’s  require¬ 
ments? 

□  Is  a  procedure  in  place  for  notifying  customers  of  unacceptable  work  and 
how  that  work  will  be  disposed  of  or  reworked  to  conform  with  require¬ 
ments? 

□  Are  reworked  designs  reviewed  for  conformance  to  the  requirements? 

4.14  Corrective  and  Preventive  Action 

4.14.1  General 

Requirement 


The  engineering  or  construction  organization  must  establish  and  maintain  docu¬ 
mented  procedures  for  implementing  corrective  and  preventive  actions  that  are 
appropriate  to  “the  magnitude  of  problems  and  commensurate  to  the  risks  en¬ 
countered”  for  the  product,  process,  and  quality  system  nonconformities.  Records 
must  be  kept  of  any  changes  to  documented  procedures  that  result  from  taking 
corrective  or  preventive  actions. 

Note:  Corrective  action  is  directed  at  eliminating  the  causes  of  actual  nonconfor¬ 
mities.  Preventive  action  is  at  toward  eliminating  the  causes  of  potential  noncon¬ 
formities. 


Guidance 


This  element  explains  what  the  organization  must  do  when  things  go  wrong.  An 
ongoing  and  viable  lessons-leamed  program  that  utilizes  records  generated  under 
4.13,  Control  of  Nonconforming  Product,  can  be  a  starting  point  for  targeting  spe¬ 
cific  problem  areas.  The  organization  should  apply  judgment  as  to  when  and 
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where  corrective  or  preventive  actions  are  required  or  even  cost-effective.  Analy¬ 
sis  of  nonconformities  can  also  be  performed  on  records  of  design  reviews,  proc¬ 
ess  monitoring  audit  observations,  and  all  other  available  feedback  methods. 
Procedures  should 

♦  establish  who  is  responsible  for  taking  corrective  action, 

♦  define  how  the  action  will  be  carried  out,  and 

♦  verify  the  effectiveness  of  the  corrective  action. 

Procedures  should  also  take  into  account  nonconformities  discovered  in  any  prod¬ 
uct  that  has  already  been  completed,  is  in  the  process  of  being  constructed,  or  has 
already  been  constructed. 

4.14.2  Corrective  Action 

When  corrective  action  is  required,  the  organization  must 

♦  effectively  handle  customer  complaints  and  nonconformity  reports, 

♦  investigate  and  analyze  the  problem  and  record  the  results 
(nonconformities  may  be  prioritized  to  determine  which  should  be  further 
investigated), 

♦  determine  the  most  effective  corrective  action  to  take,  and 

♦  ensure  that  corrective  actions  are  taken  effectively. 

Records  of  the  corrective  action  taken  should  be  established  (see  4.16,  Control  of 
Quality  Records). 

4.14.3  Preventive  Action 
Requirement 


When  preventive  action  is  required,  the  organization  must 

♦  use  all  available  information — such  as  work  processes,  internal  audit  re¬ 
sults,  quality  records,  lessons  learned,  and  customer  complaints — to  detect, 
analyze,  and  eliminate  potential  causes  of  nonconformities  in  engineering 
products,  processes,  and  quality  systems; 

♦  determine  a  method  for  preventive  action; 

♦  initiate  preventive  action  and  ensure  that  it  is  effective;  and 

♦  submit  any  relevant  information  on  actions  taken  for  management  review. 
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Records  of  the  preventive  action  taken  should  be  established  (see  4.16,  Control  of 
Quality  Records). 


Guidance 


Preventive  actions  are  intended  to  eliminate  the  causes  of  problems  in  the  organi¬ 
zation’s  processes  and  its  quality  system.  A  system  of  considering  and  applying 
lessons  learned  from  past  projects  during  the  early  planning  phase  of  new  projects 
is  one  way  to  make  sure  that  experiences  gained  from  previous  mistakes  benefit 
the  new  project. 

Guidelines  for  Conformity 

□  Are  procedures  in  place  to  identify  and  correct  nonconformities? 

□  When  work  is  found  to  be  unacceptable,  is  the  cause  of  the  deficiency 
identified? 

□  Are  preventive  measures  taken  to  preclude  recurrences  of  common  prob¬ 
lems? 

□  Are  corrective  and  preventive  actions  submitted  to  management  for  re¬ 
view? 

□  Does  the  organization  analyze  risk  versus  reward  or  return  on  investment 
to  verify  that  the  corrective  actions  taken  are  effective? 

□  Are  quality  system  elements,  processes,  and  procedure  documentation 
identified  and  changed  as  the  result  of  such  corrective  or  preventive  ac¬ 
tions? 

□  Has  the  organization  prepared  a  form  designated  for  corrective  actions, 
and  are  those  forms  logged  and  maintained? 

□  Are  the  implementation  and  effectiveness  of  preventive  actions  evaluated 
and  recorded? 

4.15  Handling,  Storage,  Packaging,  Preservation,  and  Delivery 

General 

Requirement 


The  division  must  have  procedures  to  handle,  store,  preserve,  and  package  project 
equipment  and  materials  that  are  under  its  control.  Typically,  the  procedures 
should  address  technical  and  American  Society  for  Testing  and  Materials 
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specifications  and  GFE  regulations,  maintenance  of  test  samples,  preservation 
requirements  for  long-term  storage,  etc. 

Guidance 

The  requirements  of  this  element  are  not  applicable  to  construction  divisions  that 
do  not  use  GFE  or  customer-supplied  products.  However,  it  does  apply  to  engi¬ 
neering  divisions.  Their  quality  systems  should  provide  for 

♦  secured  storage  (whether  physical  storage  or  in  electronic  format)  to  pro¬ 
tect  in-process  designs  from  theft,  deterioration,  misuse,  or  unauthorized 
revision; 

♦  good  quality  reproduction  systems  to  ensure  that  customers  get  the  quan¬ 
tity  of  designs  needed  and  in  an  acceptable  condition; 

♦  appropriate  methods  for  preserving  and  segregating  the  E&D  products 
when  they  are  under  the  engineering  organization’s  control; 

♦  protection  of  product  quality  after  final  reviews,  including  delivery  to 
customers; 

♦  appropriate  protection  against  damage,  deterioration,  or  contamination  as 
long  as  the  material  remains  the  responsibility  of  the  engineering  organi¬ 
zation; 

♦  a  clear  description  of  the  contents,  according  to  regulation  or  contract;  and 

♦  confirmation  of  packaging  effectiveness. 

If  any  aspect  of  handling,  storage,  packaging,  preservation,  or  delivery  of  E&D 
products  or  information  is  subcontracted  to  an  agent  outside  the  engineering  divi¬ 
sion,  then  the  provisions  of  4.6,  Purchasing,  apply. 

Guidelines  for  Conformity 

□  Does  the  organization  maintain  procedures  for  the  effective  handling,  stor¬ 
age,  packaging,  distribution,  and  delivery  of  in-process  and  final  products? 

□  Does  the  organization  ensure  that  its  method  of  handling  its  product  effec¬ 
tively  minimizes  the  risk  of  damage  or  deterioration? 

□  Are  the  correct  number  of  copies  of  plans  and  specifications  prepared  to 
minimize  waste  but  satisfy  needs? 

□  Are  storage  locations  (paper  files  and  electronic  storage)  secure  to  prevent 
damage,  deterioration,  or  theft? 
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□  Can  stored  plans  and  designs  be  located  and  retrieved  easily? 

□  Do  procedures  exist  for  packaging  and  delivering  the  finished  the  product 
to  customers? 

□  Are  stored  designs  periodically  inspected  to  determine  obsolescence? 

4.16  Control  of  Quality  Records 

General 

Requirement 

The  engineering  or  construction  organization  must  establish  and  document  proce¬ 
dures  for  creating,  keeping,  maintaining,  retaining,  distributing,  using,  and  dis¬ 
posing  of  all  quality  system  records  (which  may  be  in  the  form  of  paper,  electronic 
media,  or  photographs).  All  quality  records  must  be  legible  and  identifiable  to  the 
product  involved.  The  procedures  should  specify  the  people  responsible  for  the 
records  management  system,  types  of  records  to  be  maintained,  system  security, 
retrieval  procedures,  retention  intervals,  means  of  disposal,  and  a  system  for 
making  quality  records  available  to  customers  and  auditors.  Records  management 
must  be  in  accordance  with  the  Modem  Army  Record  Keeping  System. 

Guidance 


Quality  records  should,  at  a  minimum,  consist  of 

♦  internal  management  review  records, 

♦  project  management  plans  and  quality  assurance  plans  from  completed 
projects, 

♦  contract  review  agreements  (acquisition  strategy  meeting  minutes), 

♦  design  reviews  and  verifications, 

♦  internal  quality  system  audits, 

♦  measuring  and  test  equipment  calibration  results, 

♦  personnel  training, 

♦  purchase  orders, 

♦  change  orders,  and 

♦  corrective  and  preventive  actions  taken. 
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Quality  records  are  referred  to  throughout  ISO  9001.  The  purpose  of  the  records  is 
to  demonstrate  that  the  quality  system  is  effective  and  that  the  product  or  service 
meets  the  requirements — ^that  the  division  is  doing  what  it  says  it  is  doing.  The 
records  should  be  readily  accessible  and  retained  for  at  least  3  to  4  years  or  as 
specified  by  contract  or  regulation. 

Sometimes,  customers  may  require  that  the  organization  store  and  maintain,  for  a 
specified  part  of  the  operating  lifetime,  selected  records  that  attest  to  the  quality  of 
products  (particularly  for  civil  works  projects)  or  that  it  retain  design  work  that 
may  be  used  on  future  projects  or  may  be  needed  for  legal  reasons.  The  organiza¬ 
tion  must  be  able  to  provide  such  documents  to  the  customer,  as  required  under 
specific  circumstances. 

Guidelines  for  Conformity 

□  Is  a  system  in  place  for  the  identifying,  collecting,  indexing,  filing,  storing, 
maintaining,  and  disposing  of  the  organization’s  quality  records? 

□  Are  aU  records  legible,  properly  identified,  and  marked? 

□  Are  records  easily  archived  and  retrievable?  (If  controlled,  is  access  pre¬ 
vented?) 

□  Are  all  records  stored  in  such  a  way  to  preclude  damage  and  deterioration 
over  time? 

□  Are  there  policies  for  retention  and  disposal? 

□  Where  agreed  contractually,  are  records  accessible  by  the  organization’s 
customers? 

4.17  Internal  Quality  Audits 

General 

Requirement 


The  division  must  establish  and  maintain  documented  procedures  for 
comprehensive  and  systematic  (planning,  performing,  reporting,  and  foUow-up) 
quality  system  audits.  The  audits  must  be  carried  out  by  properly  trained  and 
authorized  personnel  whose  regular  responsibilities  are  outside  of  the  area  being 
audited.  The  audits  must  be  effectively  scheduled  according  to  the  status  and 
importance  of  the  activity,  and  records  of  the  findings  and  follow-up  actions  must 
be  maintained.  The  procedures  for  quality  audits  must  make  sure  that  responsible 
personnel  are  notified  of  any  deficiencies  so  that  they  can  take  timely  corrective 


A-44 


Guidelines  for  USAGE  Conformance  with  ISO  9000  Standards 


action.  The  results  of  that  corrective  action  must  be  recorded,  and  its  effectiveness 
noted  in  follow-up  internal  audits. 

Guidance 

Internal  quality  audits  are  the  mainstay  of  quality  system  conformance  and  a  pow¬ 
erful  tool  that  will  enable  continuous  improvement.  The  purposes  of  internal 
audits  are  to  ensure  that  the  organization’s  quality  system  is  working  according  to 
its  documented  plan  to  meet  customer  requirements,  to  comply  with  regulatory 
requirements,  and  to  provide  opportunities  for  improvement. 

Normally  every  area  within  the  organization  should  be  audited  at  least  annually; 
areas  consistently  having  problems  should  be  audited  more  often.  A  full  audit  plan 
should  be  developed  and  documented;  the  plan  should  include  details  for  the  cor¬ 
rective  and  preventive  action  systems.  Within  a  division,  members  of  one  branch 
may  audit  another  branch,  and  members  of  a  project  team  may  audit  another  proj¬ 
ect  team,  or  the  division  may  utilize  an  outside  organization,  such  as  the  district 
internal  review  organization.  Under  no  circumstances  should  a  branch  chief  audit 
any  area  under  his  or  her  responsibility,  nor  should  a  project  engineer  audit  a  proj¬ 
ect  under  his  or  her  control. 

Guidelines  for  Conformity 

□  Are  quality  audits  conducted  within  each  branch  or  section  to  verify 
whether  quality  activities  comply  with  established  procedures? 

□  Does  an  internal  audit  schedule  exist? 

□  Are  internal  audits  conducted  in  accordance  with  the  established  proce¬ 
dures? 

□  Are  audits  performed  by  personnel  outside  of  the  functional  activity  being 
audited? 

□  Does  the  audit  body  communicate  the  results  of  those  audits  back  to  the 
cognizant  branch  or  section  manager? 

□  Are  corrective  actions  taken  as  the  result  of  the  audits,  and  are  those 
changes  followed  up  and  are  they  determined  to  be  effective? 

□  Have  the  internal  auditors  been  adequately  trained,  preferably  through  ISO 
9000  auditor  training  programs? 

□  Are  quality  audit  records  adequately  maintained,  and  are  those  records 
kept  at  least  until  the  next  audit  is  performed? 
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□  Are  audit  results  used  as  a  basis  for  future  audits? 


4.18  Training 

General 

Requirement 

The  organization  must  have  documented  procedures  for  identifying  training  needs 
for  every  member  of  its  staff.  The  organization  must  make  sure  that  all  of  its  per¬ 
sonnel  are  effectively  trained  to  carry  out  their  responsibilities  in  a  way  that  is 
consistent  with  the  documented  quality  system.  The  documented  training  program 
should  teach  quality  and  quality  management  issues  to  all  personnel  who  directly 
affect  the  quahty  of  service,  reconcile  needed  skills  with  skills  possessed  by  every 
member  of  the  organization,  be  adequately  funded,  evaluate  the  effectiveness  of 
the  training,  and  conduct  post-training  assessments.  Records  of  employee  training 
and  training  status  must  be  kept. 


Guidance 


Training  is  essential  to  achieving  quahty  results.  The  organization  should  utihze 
the  training  system  already  estabhshed  by  the  Army’s  Individual  Training  Pro¬ 
gram  and  follow  the  broad  career  development  guidance  established  by  USAGE 
headquarters.  Within  that  program,  training  should  encompass  the  use  of,  and  un¬ 
derlying  rationale  for,  the  quality  management  approach  of  the  organization.  In  the 
training  process,  the  division  should 

♦  evaluate  the  education  and  experience  of  ah  personnel; 

♦  identify  individual  training  needs  by  job  function; 

♦  provide  the  appropriate  needed  training,  either  in-house  or  externally; 

♦  record  training  progress  and  keep  records  up  to  date  to  identify  training 
needs; 

♦  encourage  development  and  maintenance  of  industry-specific  expertise, 
particularly  when  that  expertise  is  critical  to  ongoing  project  work;  and 

♦  encourage  professional  registration  and  certifications. 

The  division  should  consider  training  people  at  all  organizational  levels  perform¬ 
ing  activities  affecting  quality.  The  training  should  include  newly  recruited  per¬ 
sonnel  and  personnel  transferred  to  new  assignments.  Executives  and  managers 
also  require  training  in  the  understanding  of  the  quality  system  and  of  the  tools 
and  techniques  needed  to  operate  the  system. 
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Guidelines  for  Conformity 

□  Is  a  procedure  in  place  and  documented  for  identifying  training  needs  for 
all  personnel  performing  activities  affecting  product  quality? 

□  Are  personnel  adequately  trained  in  skills  and  knowledge  to  perform  their 
jobs? 

□  Do  training  records  exist  for  all  employees,  and  are  they  properly  main¬ 
tained? 

□  Is  there  a  central  organizational  entity  that  maintains  training  records  and 
programs? 

□  Does  the  training  program  cover  quality  awareness  for  all  personnel? 

□  Does  the  training  program  cover  revisions  to  existing  procedures? 

4.19  Servicing 

General 

Requirement 


The  engineering  or  construction  division  must  provide  postdesign  or  postcon¬ 
struction  services  appropriate  to  the  needs  of  its  customers  (when  required  by 
contract).  On-site  technical  support  during  construction  and  dam  or  bridge  in¬ 
spection  services  are  examples  of  such  servicing.  Appropriate  records  of  support- 
related  activities  must  be  maintained  and  must  verify  that  any  required  servicing 
has  met  the  contractual  requirements. 

Note:  When  USAGE  is  the  engineer  of  record  for  any  project,  on-site  technical 
support  may  be  required  by  regulation. 


Guidance 

If  required  by  the  contract  or  regulation,  planning  procedures  for  servicing  should 

♦  clarify  technical  support  responsibilities, 

♦  plan  service  activities  (in-house  or  externally  provided), 

♦  provide  suitable  documentation  and  instructions  for  support  personnel  re¬ 
sponsible  for  on-site  support. 
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♦  provide  backup  technical  advice,  and 

♦  provide  competent,  trained  technical  support  personnel. 

Guidelines  for  Conformity 

□  Are  procedures  established  for  providing  specified  services  to  the  custom¬ 
ers? 

□  Where  required,  are  service  actions  meeting  the  expectations  of  the  cus¬ 
tomers? 

□  Are  the  appropriate  personnel  trained  for  servicing? 

□  Does  the  organization  perform  periodic  reviews  of  completed  construction 
projects  to  ensure  that  specific  elements  function  as  they  were  intended? 

4.20  Statistical  Techniques 

4.20. 1  iDENnnCATION  OF  NEED 

Requirement 


The  division  must  determine  the  need  for  statistical  techniques  in  the  various  parts 
of  the  project  management  life  cycle. 


Guidance 


Statistical  techniques  can  be  usefiil  for  nearly  every  aspect  of  an  organization’s 
operation.  Among  the  statistical  techniques  that  may  be  appropriate  are  the  fol¬ 
lowing: 


♦  Graphical  and  statistical  methods  to  help  diagnose  problems 

♦  Histograms  and  Pareto  analysis  to  help  establish  priorities  for  dealing  with 
nonconforming  products  (supports  preventive  actions) 

♦  Regression  analysis  to  improve  quantitative  models  for  a  process 

♦  Sampling  inspection  of  materials 

♦  Network  analysis  of  claims  for  effectively  negotiating  impact  of  time  de¬ 
lays 


♦  Measurement  of  quality  system  goals  and  objectives. 
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4.20.2  Procedures 

The  division  must  have  documented  procedures  for  implementing  needed  statisti¬ 
cal  techniques,  where  necessary. 

Guidelines  for  Conformity 

□  Are  the  needs  for  statistical  techniques  identified  and  used  to  determine 
design  process  capabilities,  product  characteristics,  nonconforming  prod¬ 
ucts,  test  and  inspection  controls,  process  controls,  and  customer  com¬ 
plaints? 

□  Are  the  process  variables  and  their  effects  on  the  finished  product  under¬ 
stood? 

□  Are  those  techniques  properly  used  and  reviewed  for  consistency  in  appli¬ 
cation? 

□  Are  all  statistical  procedures  controlled  and  kept  current? 

□  Are  the  appropriate  personnel  trained  in  the  use  of  the  statistical  analysis? 
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Questionnaire  for  ISO  9000  Certification  Training 
and  Effectiveness  Survey 


This  appendix  contains  the  questionnaire  sent  to  all  individuals  in  Louisville  En¬ 
gineering  and  Portland  Planning  and  Engineering  just  after  those  divisions  had 
achieved  certification  to  ISO  9001.  The  purpose  was  to  survey  managers,  engi- 
neers,  technicians,  and  administrative  employees  who  experienced  the  ISO  9000 
registration  process  to  develop  an  understanding  about  the  effectiveness  of  the 
training  they  received  and  the  benefits  to  their  organizations  of  obtaining  ISO 
9000  certification.  The  questionnaire  includes  an  area  for  respondents  to  comment 
on  aspects  of  the  certification  process  not  covered  elsewhere  in  the  questionnaire. 
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SURVEY  OF  PILOT  USAGE  DISTRICT  PERSONNEL 
ABOUT  ISO  9000  CERTIFICATION  TRAINING  AND 

EFFECTIVENESS 

This  survey  is  designed  to  obtain  information  that  will  help  the  U.S.  Army  Corps  of  Engineers 
decide  about  the  future  of  ISO  certification  in  the  Corps.  In  particular,  the  survey  asks  managers, 
engineers,  technicians,  and  administrative  employees  who  have  experienced  the  ISO  9000 
registration  process  about  the  effectiveness  of  training  they  received  and  the  benefits  to  then- 
organization  of  obtaining  ISO  9000  certification.  Your  honest  opinion  will  be  instrumental  in 
evaluating  whether  ISO  9000  certification  is  a  worthwhile  endeavor  for  the  Corps. 


Thank  you  for  your  help. 


TRAINING 


Q-1.  During  the  ISO  9000  registration  process,  training  was  accomplished  by  many 
means.  Victoria  Group,  a  consultant,  provided  supervisors  with  Implementation, 
Documentation,  and  Internal  Auditor  training  in  formal  workshops.  Supervisors  trained 
their  staff  and  staff  helped  their  peers.  How  effective  was  the  training  you  received? 

Circle  the  number  of  your  answer  for  each  source  of  training  you  received. 

If  you  did  not  receive  training  from  a  particular  source,  circle  answer  6  “Not 


Applicable.  ” 

Source 

Not 

SUghtly 

Quite 

Extremely 

Not 

of  Training 

Effective 

Effective 

Effective 

Effective 

Effective 

Auulicable 

Victoria  Group 

1 

2 

3 

4 

5 

6 

Supervisor 

1 

2 

3 

4 

5 

6 

Peer 

1 

2 

3 

4 

5 

6 

Q-2.  Some  people  also  obtained  information  about  the  ISO  9000  registration  process 
from  materials  such  as  fliers  and  manuals.  These  materials  were  produced  by  the  Victoria 
Group  and  your  district  in  bofli  hard  copy  and  electronic  media.  How  effective  were  the 
training  materials  you  received? 

Circle  the  number  of  your  answer  for  each  type  of  training  material  you  received. 
If  you  did  not  receive  a  particular  type  of  training  material  .circle  answer  6  “Not 
Applicable.  ” 


Training 

Material 

Not 

Effective 

Slightly 

Effective 

Effective 

Quite 

Effective 

Extremely 

Effective 

Not 

Annlicable 

Victoria  Group 
Manuais 

1 

2 

3 

4 

5 

6 

District  Produced  Materials: 

Manuals 

1 

2 

3 

4 

5 

6 

Fliers 

1 

2 

3 

4 

5 

6 

Electronic 
Media  (PC) 

1 

2 

3 

4 

5 

6 

ISO  9000  REGISTRATION  BENEFITS 


Q-3.  To  what  extent  do  you  agree  or  disagree  with  the  following  statements  about  the 
impact  of  the  ISO  9000  registration  process? 

Circle  the  number  of  your  answer  for  each  statement.  If  you  do  not  know  the 
answer  to  a  particular  statement,  please  indicate  “Not  Applicable.  ” 


Strongly 

Statement  Disacree 

Disagree 

Neither 
Disagree 
nor  Agree 

Agree 

Strongly 

Agree 

Not 

Annlicable 

Helped  me  better  1 

understand  what 

Ido. 

2 

3 

4 

5 

6 

Helped  my  branch  1 

better  understand 
what  we  do. 

2 

3 

4 

5 

6 

Helped  my  division  1 

better  understand 
what  we  do. 

2 

3 

4 

5 

6 

Helped  people  outside  1 

my  division 
better  understand 
what  we  do. 

2 

3 

4 

5 

6 

Improved  quality  1 

of  our  product. 

2 

3 

4 

5 

6 

Improved  customer  1 

satisfaction. 

2 

3 

4 

5 

6 

Made  the  district  1 

more  competitive. 

2 

3 

4 

5 

6 

Was  worth  the  effort  1 

2 

3 

4 

5 

6 

Q-4.  What  effect  has  ISO  9000  registration  had  on  the  efficiency  of  your  division  in 
each  of  the  following  categories? 

Circle  the  number  that  best  describes  your  opinion. 


Significantly 

Slightly 

No 

Slightly 

Significantly 

No 

Catecorv 

Less  Efficient 

Less  Efficient 

Chanse 

More  Efficient 

More  Efficient 

Opinion 

Establishing 

Customer 

Requirements 

1 

2 

3 

4 

5 

6 

Engineering 

and  Design 
Process 

1 

2 

3 

4 

5 

6 

Document 

Control 

1 

2 

3 

4 

5 

6 

Records 

Management 

1 

2 

3 

4 

5 

6 

RECOMMENDATIONS 

Q-5.  Do  you  recommend  ISO  9000  certification  for  other  USAGE  organizations? 
Circle  the  number  of  your  answer  for  each  level  of  USAGE  organization. 


NO 

LEVEL  OF  ORGANIZATION 

YES 

NO 

OPINION 

Other  organizations 
within  your  district 

1 

2 

3 

Other  USAGE  districts 

1 

2 

3 

USAGE  organizations 
above  district  level 

1 

2 

3 

PARTICIPANT  BACKGROUND  INFORMATION 

Check  the  one  appropriate  response  to  each  of  the  following  questions. 
Your  Position: 

_  Supervisor 

_ Engineer  or  Technician 

_  Administrative  or  Support 

Length  of  time  since  you  started  your  individual  ISO  9000  training 

_  More  than  12  months  1  Year 

_  6-12  months 

_  3-6  months 

_ Less  than  3  months 

_ I  have  not  received  any  ISO  9000  training. 


COMMENTS 


Would  you  like  to  comment  on  your  answers  to  any  of  these  questions  or  provide  other 
suggestions  about  the  ISO  9000  process  or  your  experiences  with  it  which  would  help 
HQS  USAGE  to  decide  about  its  future  use  in  the  Corps?  If  so,  please  use  this  space  (and 
additional  sheets  as  necessary)  for  that  purpose. 


Results  from  this  questionnaire  will  be  included  as  a  major  input  to  the  pilot  study  report 
providing  recommendations  to  Headquarters  USAGE,  Copies  of  the  report  will  be 
provided  to  your  office  when  complete.  Your  contribution  to  this  survey  and  your  effort 
in  this  pilot  program  is  greatly  appreciated. 


Appendix  C 

Results  of  ISO  9000  Certification  Training  and 
Effectiveness  Survey 


LMI  received  completed  questionnaires  about  the  ISO  9000  certification  training 
and  effectiveness  from  218  respondents.  Of  them,  131  were  from  Louisville 
Engineering,  and  87  were  from  Portland  Planning  and  Engineering.  As  Figure  C-1 
shows,  75  percent  of  the  respondents  were  engineers  or  technicians.  More  than  60 
percent  of  the  respondents  indicated  that  they  had  received  their  ISO  9000  training 
6  months  or  more  before  they  responded  to  our  questionnaire. 

Figure  C-1.  Distribution  of  Positions  Held  by  Respondents 


We  entered  the  responses  into  an  Access  database,  then  imported  the  data  into 
SPSS  for  Windows  (Version  7.5)  for  statistical  analysis.  For  each  of  the  five  key 
survey  questions,  we  developed  distributions  for  three  groupings  of  respondents: 
all  respondents;  respondents  by  position  (supervisor,  engineer  or  technician,  and 
administrative  or  support  personnel);  and  respondents  by  time  since  they  had 
received  ISO  9000  training. 

In  our  analyses,  we  used  measures  of  central  tendency  to  define  the  nature  of  the 
various  distributions.  These  measures  include  the  mean,  median,  standard 
deviation,  and  variance.  We  also  applied  measures  of  kurtosis  and  skewness  to  the 
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histograms  to  develop  an  understanding  of  the  normal  curves,  or  the  shape  of  the 
distributions.  To  understand  the  most  frequently  chosen  response  for  each 
question  by  each  of  the  three  groupings  of  respondents,  we  used  a  mean  ranking. 
In  selected  cases,  we  used  one-way  analysis  of  variance  (ANOVA)  to  determine 
the  extent  to  which  the  difference  in  the  mean  values  within  groups  were 
significant. 

This  appendix  summarizes  the  results  of  our  analyses.  The  results  are  instructive 
in  that  they  illustrate  perceived  benefits  and  efficiencies  achieved  as  a  result  of 
certification  and  rate  the  effectiveness  of  the  training  techniques  and  materials 
used  during  the  process.  However,  given  the  distribution  of  respondents  by 
position,  the  survey  results  are  heavily  influenced  by  engineers  and  technicians. 

Key  Results 

The  most  important  questions  (Questions  3  and  4)  concern  the  extent  to  which 
respondents  agreed  witii  statements  about  the  impact  of  ISO  9000  registration  on 
the  organization  and  the  effect  that  ISO  9000  registration  has  had  on  certain 
aspects  of  organizational  efficiency.  When  analyzing  the  answers  to  these 
questions,  we  took  a  mean  ranking  of  responses  to  determine  which  statements 
respondents  viewed  most  favorably: 

♦  Of  eight  possible  choices  in  Question  3,  all  respondents  agreed  most 
favorably  with  the  statement  that  the  ISO  9000  registration  process 
“helped  my  division  better  understand  what  we  do.”  The  distribution  for 
this  response  is  shown  in  Figure  C-2. 

♦  When  considering  the  question  about  the  effect  ISO  9000  registration  has 
had  on  the  efficiency  of  fom  categories  within  the  division,  respondents 
rated  document  control  as  having  experienced  the  greatest  improvements. 
The  distribution  for  this  response  is  depicted  in  Figure  C-3. 

♦  In  every  category  of  Questions  3  and  4,  supervisors  responded  with  greater 
approval  with  regard  to  the  perceived  benefits  of  ISO  9000  registration  and 
were  more  optimistic  about  the  extent  to  which  the  ISO  9000  registration 
process  helped  die  organization  achieve  efficiencies. 
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Figure  C-2.  Impact  of  ISO  Registration  Process — 
Highest  Ranked  Statement 
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Helped  My  Division  Better  Understand  What  We  Do 


1=:strongly  disagree  <— >  5=strongly  agree 


Figure  C-5.  Effect  of  ISO  9000  Registration  on  Efficiency — 
Highest  Ranked  Efficiency  Improvement 
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Document  Control 

1=significantly  less  efficient  < — >  5=significantly  more  efficient 
The  remaining  sections  discuss  the  results  in  more  detail. 


Perceived  Benehts 

Question  3  focused  on  the  extent  to  which  respondents  agreed  with  several 
statements  provided  in  the  survey.  Table  C-1  displays  the  mean  responses  to  these 
statements,  split  by  respondent  groupings  (supervisors,  engineers  and  technicians, 
and  administrative  and  support).  In  this  table,  we  used  a  one-way  ANOVA  to 
identify  the  statements  for  which  the  difference  in  responses  among  the  groups 
was  significant.  Several  of  the  statements  have  a  less  than  1-in-lOO  chance  (0.01) 
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of  having  the  same  mean  score.  Gtenerally,  scores  of  more  than  .05  indicates  that 
there  is  no  significant  difference  in  mean  scores. 


Table  C-1.  Mean  Ranking  of  Statements  About  the  Impact  of  the  ISO  9000  Registration  Process 


Overall 

Supervisors 

Engineers/ 

technicians 

Administrative/ 

support 

One-way 

ANOVA 

Statement 

Rank 

Mean 

Rank 

Mean 

Rank 

Mean 

Rank 

Mean 

F 

Sig. 

Helped  my  division 
better  understand 
what  we  do 

1 

3.07 

1 

3.93 

■ 

2.91 

1 

3.44 

9.40 

.000 

Was  worth  the  effort 

3.06 

3.93 

2.93 

3.00 

6.03 

.001 

Made  the  district 
more  competitive 

3.06 

3.64 

H 

2.94 

3.13 

2.85 

.038 

Improved  quality  of 
our  product 

4 

3.03 

4 

3.68 

1 

2.94 

4 

3.12 

4.31 

.006 

Helped  my  branch 
better  understand 
what  we  do 

5 

2.92 

3 

3.75 

5 

2.78 

5 

3.06 

6.62 

.000 

Improved  customer 
satisfaction 

6 

2.84 

7 

3.46 

7 

2.72 

3 

3.12 

4.60 

.004 

Helped  me  better 
understand  what  1  do 

7 

2.83 

6 

3.61 

8 

2.71 

8 

3.00 

5.56 

.001 

1 

Helped  people 
outside  my  division 
better  understand 
what  we  do 

8 

2.79 

8 

3.07 

6 

2.74 

6 

3.00 

1.30 

.275 

Notes:  F  =  F-statistic;  sig.  =  significance. 


Perceived  Efficiencies  Achieved 

Question  4  focused  on  the  extent  to  which  ISO  9000  registration  had  an  effect  on 
efficiencies  in  selected  categories.  Table  C-2  displays  the  mean  responses  to  these 
categories,  split  by  respondent  groups.  Again,  we  used  a  one-way  ANOVA  to 
identify  the  categories  for  which  responses  among  groups  differed  significantly. 
As  the  table  shows,  responses  differed  from  group  to  group,  but  the  difference  in 
mean  scores  for  document  control  and  records  management  are  not  significant. 


C-4 


Results  of  ISO  9000  Certification  Training  and  Effectiveness  Survey 


Table  C-2.  Mean  Ranking  of  Effect  of  the  ISO  9000  Registration  Process  on  Efficiency 


Overall 

Supervisors 

Engineers/ 

technicians 

Administrative/ 

support 

One-way 

ANOVA 

Statement 

Rank 

Mean 

Rank 

Mean 

Rank 

Mean 

Rank 

Mean 

F 

Sig. 

Document  control 

n 

2 

4.15 

2 

3.72 

1 

3.93 

Records 

management 

H 

1 

4.15 

1 

3.73 

B 

3.69 

Establishing  customer 
requirements 

B 

3.42 

B 

3.96 

B 

3.36 

3.08 

6.65 

.000 

Engineering  and 
design  process 

B 

3.13 

B 

3.65 

B 

3.07 

B 

3.25 

4.56 

.004 

Notes:  F  =  F-statistic;  sig.  =  significance. 


Other  Important  Results 

While  the  more  critical  aspects  of  the  survey  are  covered  in  Questions  3  and  4,  a 
few  other  observations  should  be  made: 

♦  When  asked  about  the  effectiveness  of  ISO  9000  training,  respondents 
rated  three  sources.  A  mean  ranking  indicated  that  the  most  effective 
source  of  training  was  the  Victoria  Group.  Supervisors  and  peers, 
respectively,  were  the  second  and  third  most  effective  sources  of  training. 

♦  Respondents  also  were  asked  about  the  effectiveness  of  four  types  of  ISO 
9000  training  materials.  The  mean  ranking  indicated  that  the  most 
effective  were  Victoria  Group  manuals.  The  other  types,  in  order  of 
effectiveness,  were  district  electronic  media,  district  manuals,  and  district 
fliers. 

Overall,  respondents  were  evenly  divided  when  asked  whether  they  would 
recommend  ISO  9000  certification  for  other  USAGE  organizations  (Question  5). 
However,  analysis  of  the  responses  by  position  of  the  respondents  showed  that 
supervisors  strongly  recommended  certification  for  other  USAGE  organizations. 
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Appendix  D 

Survey  of  Sites  in  Pilot  Program 


Introduction 

In  February  1998,  LMI  sent  questionnaires  to  the  four  engineering  and  two 
construction  divisions  participating  in  the  ISO  9000  pilot  program.  The 
questionnaire  had  24  open-ended  questions  covering  the  entire  process  used  to 
prepare  for  ISO  9000  registration,  from  the  organization  phase  to  the  final  external 
audit  for  registration  (if  registration  had  been  achieved  by  the  survey  date). 

In  this  appendix,  we  provide  the  responses  made  by  Savannah  Construction, 
Louisville  Construction,  Louisville  Engineering,  and  Portland  Engineering  to  our 
questions.  (Savannah  Engineering  and  Kansas  City  Engineering  did  not  respond  to 
Ae  questionnaire).  The  responses  have  been  edited  slightly  for  style,  clarity, 
brevity,  and  grammar. 

Questions  and  Responses 

1.  When  did  you  start  preparing  for  ISO  9000  registration? 

Savannah  Construction:  The  first  training  effort  to  prepare  for  implementing  ISO 
9000  in  the  Savannah  District  took  place  in  February  1996.  Serious  effort  on 
implementing  ISO  9000  began  in  May  1996.  The  goal  in  Savannah  was  to 
implement  the  quality  standards,  not  necessarily  achieve  registration.  We  did  not 
understand  achieving  registration  to  be  the  primary  goal  from  our  communications 
with  USACE  headquarters,  LMI,  or  the  Victoria  Group. 

Louisville  Construction:  In  the  spring  of  1995. 

Louisville  Engineering:  Our  first  training  (ISO  Implementation  Workshop)  was 
conducted  in  March  1995. 

Portland  Engineering:  Unofficially  in  June  1995  (prior  to  HQ  funding);  officially 
in  October  1995. 

2.  When  did  you  achieve  registration?  If  not  yet  registered,  when  do  you  project  achieving 
registration? 


Savannah  Construction:  Anticipate  being  ready  for  an  extrinsic  audit  in  June 
1998. 
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Louisville  Construction:  No.  Preassessment  audit  will  be  performed  in 
February/March  1998.  Pending  the  outcome  of  this  audit,  the  registration  audit 
may  be  scheduled  in  June/July  1998. 

Louisville  Engineering:  The  Louisville  District  Engineering  Division  was 
registered  on  27  March  1997. 

Portland  Engineering:  August  1997. 

3.  Thinking  of  the  elapsed  time  between  1  and  2  above,  how  much  calendar  time  was  spent 
between  three  broad  categories  of  activity:  (1)  organizing  (figuring  out  what  you  had  to  do),  (2) 
reviewing  processes/procedures  and  preparing  documentation,  and  (3)  implementing  your  new 
quality  management  system? 

Savannah  Construction:  (1)  6  months,  (2)  12  months,  (3)  7  months  to  allow  two 
rounds  of  internal  audits. 

Louisville  Construction:  (1)  Approximately  14  months,  (2)  Approximately  17 
months,  (3)  Approximately  2  months. 

Louisville  Engineering:  (1)  6  months,  (2)  10  months,  (3)  3  months. 

Portland  Engineering:  (1)  7  months,  (2)  5  months,  (3)  3  months. 

4.  What  was  the  division’s  organizational  structure  at  the  time  you  began?  Has  it  changed 
since?  (Please  attach  organization  chart.)  If  so,  did  the  change  result  from  the  process  analysis 
during  preparation  of  your  ISO  9000  documentation?  What  is  your  FTE  authorization? 

Savannah  Construction:  Organization  has  not  changed  significantly  in  this  time 
period  other  than  disestablishment/establishment  of  field  offices  because  of 
workload  changes.  I  would  not  anticipate  any  significant  organizational  changes 
because  of  ISO  9000.  There  are  simply  too  many  other  factors  involved.  FTE 
authorization  is  253,  military  and  civil. 

Louisville  Construction:  Fewer  people,  but  organizational  structure  is  the  same. 
(Organizational  chart  is  attached.)  There  have  been  no  changes  to  organizational 
structure  because  of  ISO.  Our  FTE  authorization,  based  upon  affordability,  is 
43.99  for  civil  works  and  1 19.48  for  military  (as  of  20  Dec  97). 

Louisville  Engineering:  Our  current  FTE  is  approximately  220.  Our  organization 
has  changed,  but  the  primary  motivation  for  change  was  not  the  result  of  ISO.  We 
are  in  the  process  of  focusing  on  a  Quality  Officer  position. 

Portland  Engineering:  A.)  No  change  in  the  physical  makeup  of  the 
organizational  chart.  B.)  One  branch  was  moved  to  another  division  (move  was 
predetermined,  not  because  of  ISO  implementation).  C.)  No  changes  occurred  as  a 
result  of  ISO  implementation/documentation.  D.)  Approximately  160. 
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5.  Tell  us  how  your  division  approached  the  assignment,  i.e.,  did  you  approach  the  task  of 
preparing  for  registration  as  a  project? 

Savannah  Construction:  We  initially  approached  the  task  of  implementing  ISO 
9000  as  a  project.  It  was  simply  another  thing  to  be  done  and  was  not  embraced  as 
a  fundamental  change  in  our  business  process. 

Louisville  Construction:  Yes.  The  management  team  was  formed  and  a 
management  representative  was  appointed  by  the  division  chief.  Training  on  what 
ISO  is  followed  formulation  of  team.  Assignments  for  drafting  the  various 
procedures  was  given  to  groups  of  employees.  Training  of  internal  auditors  was 
provided  last. 

Louisville  Engineering:  We  relied  very  heavily  on  the  services  of  the  Victoria 
Group  to  lead  us  through  the  process,  but  we  established  a  schedule  to  obtain 
registration,  identified  the  major  milestones  necessary  to  accomplish  it,  and 
handled  the  process  similarly  to  the  development  of  a  project. 

Portland  Engineering:  With  the  help  of  a  consultant  (Victoria  Group),  an 
approach  to  an  business  operations  model  was  formulated  and  then  implemented. 

6.  What  was  the  management  structure  for  the  project  (DE  down  to  management 
representative)? 

Savannah  Construction:  The  management  representative  was  tasked  with 
implementing  ISO  9000.  There  was  little  daily  management  involvement  above 
this  level. 

Louisville  Construction:  District  engineer,  division  chief,  management  team,  and 
management  representative  constituted  the  order  of  management  structure  set  in 
place  for  implementation  of  ISO. 

Louisville  Engineering:  We  identified  a  management  representative  (assistant 
chief  of  engineering),  program  manager  (chief  of  QA  section),  and  the  remaining 
membership  of  the  management  team  (chief  engineering,  branch  chiefs,  and  chief 
of  management  support  section).  There  was  no  role  of  district  staff  outside  of 
Engineering,  with  the  exception  of  the  Internal  Review  Office,  which  reviewed 
our  procedures. 

Portland  Engineering:  Management  structure  was  primarily  confined  solely  to 
Planning  and  Engineering,  with  the  management  representative  being  appointed 
from  one  of  the  existing  managers. 


7.  What  was  the  organizational  position  of  the  person  designated  as  the  management 
representative?  Was  this  particular  person  selected  based  on  position,  ability,  availability,  or 
what? 


Savannah  Construction:  Organizational  position  is  that  of  section  chief,  GM-13. 
The  factors  indicated  to  be  involved  in  the  selection  were:  position  (current 
position  already  involved  significant  duties  with  direct  ISO  9000  appUcation), 
availability  and  individual’s  ability. 

Louisville  Construction:  The  management  representative  is  the  chief  of  quality 
assurance.  Since  ISO  is  a  quality  system,  logically  the  position  went  to  the  person 
responsible  for  quality  in  the  organization. 

Louisville  Engineering:  Selection  was  based  on  position  and  ability.  We  wanted 
an  individual  from  management  high  enough  in  die  organization  to  create  die 
proper  image  for  the  overall  effort.  The  total  process  required  the  efforts  of 
numerous  staff  during  the  development  phase  and  the  cooperation  of  all  staff 
during  the  implementation  phase,  and  without  the  proper  emphasis  at  the  senior 
management  level,  we  did  not  feel  we  could  succeed. 

Portland  Engineering:  Chief,  Technical  Resources  Branch.  Supervisory  position, 
ability  and  availability,  no  lower  than  branch  chief  level. 

8.  Did  anyone  work  on  the  task  full  time?  If  so,  what  was  their  job  (position)  before 
undertaking  this  assignment? 

Savannah  Construction:  There  was  no  one  assigned  to  work  on  ISO  9000  full 
time  for  the  entire  period.  There  were  a  couple  of  2-  to  3 -month  periods  when  a 
person  was  assigned  to  work  on  this  task  solely. 

Louisville  Construction:  No.  However,  separate  drop  codes  were  established  to 
capture  employee  efforts  on  ISO. 

Louisville  Engineering:  No  one  worked  full  time  on  this  initiative. 

Portland  Engineering:  Yes,  ISO  coordinator;  Chief,  Contract  Administration 
(Construction). 

9.  What  team  structure  did  you  put  in  place  to  prepare  the  documentation? 

Savannah  Construction:  Basic  structure  was  developed  by  the  management 
representative.  Individual  procedures  in  support  of  ISO  elements  were  assigned  to 
several  task  groups.  SOPs  (level  3  documents)  were  assigned  to  CD  offices  that 
were  already  proponents  for  the  various  processes. 

Louisville  Construction:  The  management  team  members  are  all  supervisors  from 
the  management  positions  in  the  division.  Teams  for  writing  procedures  were 
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hand  picked  from  all  employees  in  the  division  based  upon  their  knowledge  of  the 
subject.  A  least  one  existing  supervisor  (not  necessarily  on  the  management  team) 
was  placed  on  each  procedure  writing  team  as  a  leader.  However,  one  procedure 
writing  team  contained  no  supervisors  and  worked  well. 

Louisville  Engineering:  Following  our  gap  analysis,  we  decided  what  additional 
guidance  documents  were  going  to  be  required.  We  identified  the  need  for 
additional  guidance/procedures  in  21  different  areas.  A  leader  for  the  development 
of  each  new  quality  procedure  document  was  selected  from  the  management  team 
and  a  quality  action  team  was  assembled  to  prepare  the  required  documents. 
Several  teams  were  assigned  to  cover  multiple  areas  when  there  was  a  natural 
relationship  between  those  areas.  Total  staff  involved  in  the  preparation  of  these 
documents  was  approximately  40. 

Portland  Engineering:  No  formal  team  stracture  was  used  to  prepare  the 
documentation.  Branch  chiefs  and  a  few  other  key  people  performed 
brainstorming  and  preparation  of  the  quality  manual. 

10.  How  many  groups/people  actually  worked  on  drafting  materials  for  the  quahty  manual? 
Did  you  have  an  editor  and  production  team  preparing  final  documents? 

Savannah  Construction:  There  were  approximately  15  people  who  worked  on 
drafting  procedures.  There  was  no  editor/production  team  set  up.  There  was  an 
approval  process  developed  and  the  management  representative  reviewed  and 
signed  off  on  every  document,  not  necessarily  for  content  but  to  ensure  that  it  fit 
into  the  quality  system. 

Louisville  Construction:  Fifteen  groups  of  people  worked  on  drafting  the 
procedures.  The  number  of  people  working  on  writing  the  procedures  was  42 
within  the  division  and  1  outside  the  division.  The  management  representative, 
with  the  help  of  three  consultants  and  one  contracted  employee,  edited  and 
produced  the  controlled  ISO  procedures. 

Louisville  Engineering:  See  response  to  Question  9.  One  team  of  approximately  4 
members  prepared  the  quality  manual.  A  similar  process  was  used  for  each  EQP 
developed.  During  the  development  phase,  procedures  were  being  reviewed 
monthly  by  Victoria  Group/LMI.  Mr.  Jinks  came  to  the  Louisville  District  on 
numerous  occasions  to  review  the  documents.  In  addition  Mr.  Hawkins  had  the 
staff  of  LMI  perform  an  editorial  review  of  the  documents.  When  we  believed  the 
documents  were  near  final  form,  each  member  of  the  management  team  reviewed 
the  entire  package. 

Portland  Engineering:  Over  a  dozen  personnel  were  involved  in  generating  the 
quahty  manual.  In  a  manner  of  speaking,  the  management  representative,  ISO 
coordinator,  and  staff  reviewed,  edited,  and  commented. 
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1 1 .  Please  explain  your  document  control  system  for  the  preparation  of  the  quality  manual 
and  work  instructions. 

Savannah  Construction:  See  procedure  CQP  5-01  and  excerpts  from  the  quality 
manual,  which  are  attached.  These  documents  can  also  be  viewed  at 
www.sas.usace.army.mil/cd/cdnet. 

Louisville  Construction:  The  quality  system  has  been  placed  on  the  division’s 
intranet  and  will  be  maintained  by  one  of  the  division’s  branches.  This  will  be  the 
“controlled”  copy. 

Louisville  Engineering:  The  preparation  of  the  quality  manual  was  accomplished 
by  one  of  the  quality  action  teams,  and  Mr.  Sommerville  served  as  the  leader  of 
this  effort.  The  preparation  of  all  EQPs  were  handled  in  a  similar  fashion.  As 
documents  were  nearing  completed  status,  they  were  made  available  on  our  LAN 
system,  but  each  page  of  each  document  carried  the  following  footnote: 
(UNCONTROLLED  DOCUMENT  REVIEW  COPY  ONLY).  Not  until  the 
document  was  approved  and  subjected  to  the  signature  process  described  in  EQP- 
5-01  (Procedure  for  the  Preparation  and  Administration  of  Procedures)  was  this 
footnote  removed.  Altogether  we  generated  a  total  of  36  EQPs  (now  37)  in 
addition  to  the  quality  manual.  Numerous  other  existing  documents  were 
converted  into  work  instructions  with  a  similar  but  less  stringent  process.  This 
process  is  also  described  in  EQP-5-01.  All  new  documents,  as  well  as  any  changes 
that  have  resulted  since  initial  implementation,  are  accomplished  through  a 
process  we  call  implementation  memorandums,  which  are  e-mailed  to  all 
Engineering  staff.  These  memorandums  serve  as  official  notification  of 
implementation.  Records  of  official  implementation  of  the  current  version  of 
every  internally  generated  document  can  be  traced  to  an  implementation 
memorandum. 

Portland  Engineering:  The  quality  manual  consists  of  two  parts  (Part  1  Policy  and 
Part  2  Procedures),  and  it  applies  throughout  Planning  and  Engineering.  A  policy 
statement  was  created  for  each  ISO  9001  element;  procedures  were  drafted  to 
document  the  business  operating  system  as  we  know  it.  Policy  is  cross-referenced 
to  associated  procedures.  Work  instructions  were  left  for  individual  sections  or 
branches  to  create  and  maintain  on  their  own  and  do  not  apply  to  the  rest  of  the 
division.  One  copy  of  the  quality  manual  was  assigned  to  each  branch  and  section 
chief  and  to  each  internal  auditor. 

12.  From  where  within  the  district’s  organization  did  you  select  your  internal  auditors? 

Savannah  Construction:  All  auditors  are  members  of  the  Construction  Division. 

Louisville  Construction:  One  of  the  division’s  branches  was  already  performing 
quality  assurance  team  visits  to  field  offices.  This  branch  now  performs  the 
internal  audits.  So,  an  existing  system  already  in  place  was  adopted.  For  the 
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division  office,  the  district’s  Internal  Review  Office  performs  the  internal  review 
of  the  division  office. 

Louisville  Engineering:  We  utilize  our  Internal  Review  Office  for  this  effort. 

From  a  district  organizational  structure,  this  office  is  on  the  same  level  as  the 
Engineering  Division  or  any  other  division  within  the  Louisville  District.  We  also 
had  some  of  Engineering’s  staff  trained  in  auditing,  but  have  not  yet  officially 
used  them  to  perform  this  function.  We  plan  to  assist  Construction  in  this  effort. 

Portland  Engineering:  Approximately  one  individual  per  section. 

13.  How  did  you  train  your  entire  work  force  to  become  familiar  with  the  ISO  9000  program 
and  their  part  in  it? 

Savannah  Construction:  We  put  together  a  training  program  and  trained  all  in  the 
Construction  Division.  Training  materials  can  also  be  viewed  at  our  Web  site. 

Louisville  Construction:  The  division  chief  and  his  management  representative 
went  to  each  field  office  location  and  held  4  hours  of  training  or  orientation  on 
ISO. 

Louisville  Engineering:  We  provided  numerous  ISO  orientation  sessions, 
provided  several  other  training  sessions,  and  utilized  e-mail  communication  to 
educate  and  explain  the  overall  process.  Numerous  e-mail  messages  in  the  context 
of  training  were  issued  to  Engineering  staff.  We  asked  all  staff  to  demonstrate 
their  support  for  the  overall  initiative  by  signing  a  poster-sized  copy  of  our  quality 
policy.  Basically  we  ran  an  advertisement  blitz  during  the  entire  process.  In 
addition  we  furnished  a  brochure  to  each  employee  to  explain  the  process  and  give 
questions/answers. 

Portland  Engineering:  Town  hall  (staff)  meetings  with  the  division  chief, 
management  representative,  and  ISO  coordinator.  Numerous  meetings  with 
individuals,  sections,  and  branches.  Internal  auditor  training  and  internal  audits 
also  served  to  familiarize  staff  with  ISO-based  requirements. 

14.  How  much  money  (direct  and  indirect)  did  you  spend  preparing  for  registration? 

Savannah  Construction:  Approximately  $350,000,  give  or  take.  Some  costs  may 
have  been  lost  during  CEFMS  blackout.  Also,  this  figure  includes  costs  for 
revamping  our  existing  SOPs,  which  we  would  have  had  to  do  whether  we  were 
implementing  ISO  or  not. 

Louisville  Construction:  The  final  costs  are  not  in.  It  is  expected  that  our  cost  will 
round  out  somewhere  near  to  $125,000.  Costs  paid  to  the  consultants  sponsored 
by  OCE  are  unknown. 
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Louisville  Engineering:  The  information  presented  below  represents  a  summary  of 
various  reports  and  requests  for  information  that  Louisville  district  has  provided 
to  our  higher  command  during  the  entire  process;  it  also  provides  additional 
information  covering  several  other  questions  contained  in  your  request. 

A.  Schedule  for  implementation;  progress  made. 

ORL’s  schedule  for  implementation  is  as  follows: 


Training  started 

Mar  1995 

Gap  evaluation 

Jun  1995 

Fill-in  gap  training 

Nov  1995 

Up  and  running 

Apr  1996 

Officially  implemented 

Jul  1996 

Documentation  review 

Sep  1996 

Preassessment  audit 

Oct  1996 

Certification  audit 

Jan  1997 

Received  certificate 

Mar  1997 

B.  Cost  to  date  for  implementation  and  estimated  cost  to  complete. 

Labor  costs  above  normal  costs  of  operation: 

FY95  $25,000 

FY96  $150,000 

FY97  $15,000 

Total  labor  cost  $ 1 90,000 

Registrar  contract  $25,000 

C.  Lessons  learned  to  date  including  a  best  estimate  of  value  that  will  be 
added  by  implementation  of  ISO  9000. 

After  having  gone  through  this  process,  we  feel  very  positive  about  the 
experience.  While  we  thought  we  were  doing  quality  work  before,  we 
discovered  numerous  areas  that  needed  improvement  and  have  already 
implemented  many  of  these  initiatives. 

While  the  cost  was  not  cheap,  we  feel  the  benefits  wiU  far  exceed  the 
expense.  When  you  are  forced  to  look  at  your  overall  operating  system  in 
comparison  to  the  20  elements  of  the  9001  standard,  it  becomes  painfully 
obvious  how  inconsistent  and  ntistake  prone  your  normal  procedures  have 
been.  ISO  9000  is  just  good  business  practice.  It  does  not  represent  nice- 
to-have  features  but  basically  represents  the  minimum  requirements 
needed  to  consistently  produce  a  high-quality  product.  An  argument  can  be 
made  that  registration  is  not  required  because  the  same  benefits  can  be 
achieved  by  just  implementing  the  procedures.  While  in  theory  this  may  be 
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true,  the  Louisville  District  has  experienced  great  benefits  from  having 
gone  through  the  process  and  facing  the  very  stringent  challenge  of  passing 
the  registration  process.  Merely  trying  to  implement  the  procedures, 
without  planning  to  seek  registration,  would  make  this  process  just  another 
management  program  with  no  real  incentive  for  success.  Registration 
forces  all  district  staff  to  endorse  the  goal  and,  once  obtained,  to  maintain 
it.  Follow-up  audits  ensure  that  your  procedures  are  continually  followed. 
They  will  keep  your  efforts  honest  and  will  not  allow  regression  into  old 
habits. 

D.  A  statement  of  how  ISO  9000  has  (or  will)  mesh  with  the  present  system 
of  QA  being  performed  by  MSCs. 

ISO  9000  does  not  contradict,  or  negate  in  any  way,  current  QC/QA 
activities.  In  some  cases,  9001  requirements  go  beyond  MSC 
requirements,  but  these  only  reinforce  the  goal  of  producing  high-quality 
products.  The  only  problem  we  have  had  in  trying  to  mesh  the  two  systems 
has  been  in  terminology  and  definitions.  We  have  to  be  careful  in 
terminology  used  in  our  procedures  and  work  instructions. 

E.  Any  general  comments. 

Do  not  try  to  short-cut  the  effort  required  to  develop  the  documentation 
requirements  of  ISO  9001.  Without  going  through  this  step,  you  will  never 
get  a  full  handle  of  your  organization’s  quality  operating  system,  and  the 
whole  process  will  become  just  a  paper  exercise  without  the  learning 
experience.  We  cannot  overemphasize  the  importance  of  this  process  and 
why  it  is  necessary.  Until  ORL  went  through  this  process  (which  included 
many  cycles  of  trial  and  error),  we  never  really  fully  understood  what  ISO 
9000  was  all  about.  It  is  this  process  that  will  turn  unbelievers  into 
proponents  of  the  system.  ISO  9001  is  a  minimum  quality  system.  It  is  just 
another  piece  in  our  TQM  initiative.  Without  this  in  place,  it  is  impossible 
to  see  how  we  can  strive  for  an  APIC  type  organization.  Our  recent  APIC 
“gap”  analysis  has  indicated  this. 

Portland  Engineering:  Approximately  $264,000. 

What  problems  have  you  encountered  since  achieving  registration? 

Savannah  Construction:  Not  applicable. 

Louisville  Construction:  Not  applicable. 

Louisville  Engineering:  There  is  a  tendency  to  relax  after  the  flurry  of  activity 
required  to  obtain  the  registration.  The  natural  tendency  is  to  focus  on  obtaining 
registration  rather  than  on  the  creation  of  and  adherence  to  a  quahty  operating 
system.  It  is  difficult  to  keep  people  motivated  toward  continual  compliance,  but 
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we  are  making  progress.  At  first  there  were  problems  dealing  with  DL,  but  these 
problems  have  been  corrected  and  now  DL  is  using  some  of  our  ISO  procedures. 

Portland  Engineering:  Maintaining  momentum  at  a  high  energy  level  of 
involvement,  interest,  and  support. 

16.  How  have  you  kept  your  internal  auditors  trained  and  effective? 

Savannah  Construction:  We  have  had  follow-up  audit  training  (given  by  Paul 
Jenks)  and  have  gained  experience  by  conducting  15  internal  audits  to  date. 

Louisville  Construction:  Internal  auditors  were  trained  by  the  Logistics 
Management  Institute,  McLean,  VA. 

Louisville  Engineering:  Internal  audits  are  accomplished  through  our  Internal 
Review  Office.  Individual  auditors  are  each  professionally  trained.  Other  than 
expertise  maintained  through  the  frequent  audits  that  they  continue  to  provide  to 
the  Engineering  Division,  no  additional  training  specifically  tailored  to  ISO 
requirements  is  being  conducted. 

Portland  Engineering:  Yes  (as  well  as  OJT).  A  mock  preassessment  was 
conducted  by  our  consultant  to  disclose  any  large  weaknesses  in  our  internal  audit 
system. 

17.  How  have  you  kept  the  division’s  work  force  trained  in  the  application  of  your  ISO 
program  (assuming  new  hires,  the  usual  staff  rotation)? 

Savannah  Construction:  Part  of  our  employee  indoctrination  routine  is  ISO  9(XX) 
orientation.  This  is  covered  in  our  CQP  18-02.  The  materials  located  on  our  Web 
site  are  used  by  the  supervisors  to  conduct  this  orientation. 

Louisville  Construction:  It  is  the  responsibility  of  the  supervisors  to  familiarize 
dieir  new  hires  on  die  ISO  quality  system. 

Louisville  Engineering:  We  provide  ISO  orientation  to  new  staff.  We  perform 
quarterly  project  reviews  of  all  ongoing  projects,  and  this  serves  as  a  good  forum 
for  reinforcement  of  ISO  requirements  and  allows  management  staff  to  evaluate 
the  level  of  understanding  and  practice  of  ISO  requirements.  Additional  training 
or  refresher  training  may  be  scheduled  if  warranted.  As  also  covered  in  response 
to  Question  13,  we  utilize  our  e-mail  system  to  notify  all  staff  of  new  or  revised 
procedures.  During  orientation  training  for  new  employees,  some  of  our 
previously  trained  staff  have  chosen  to  also  attend  as  a  refresher. 

Portland  Engineering:  Yes,  new  hires  included.  Have  recentiy  initiated  weekly 
sessions  with  each  office  to  discuss  quality  manual  changes  and  developments. 
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18.  If  you  had  it  to  do  all  over  again,  what  changes  would  you  make  to  the  process  you  used 
to  prepare  for  registration? 

Savannah  Construction:  One  or  more  bodies  would  be  assigned  full  time.  There 
would  also  be  a  greater  commitment  of  resources.  Funds  and  FTEs  would  be 
requested  to  implement  ISO  9000  without  robbing  from  our  ongoing  work.  Would 
also  involve  the  district  commander. 

Louisville  Construction:  It  is  believed  most  would  simply  say,  finding  more  time 
to  work  on  ISO  would  have  made  the  process  go  faster.  However,  time  is  a 
premium. 

Louisville  Engineering:  We  would  probably  use  a  very  similar  process.  There  may 
be  some  training  costs  that  could  have  been  eliminated,  but  overall  the  process  we 
followed  worked  well. 

Portland  Engineering:  Make  better  use  of  the  consultant  (Victoria  Group);  initiate 
weekly  discussions  at  an  earher  stage  of  quaUty  system  development;  “live  the 
system”  longer;  and  perform  mock  audits  by  the  consultant  before  any  official 
third-party  assessment. 

19.  Do  you  believe  that  it  would  be  practical  for  more  than  one  division  within  a  district  to 
work  toward  achieving  registration  at  the  same  time  under  one  certificate? 

Savannah  Construction:  Only  would  work  if  the  district  commander  was  strongly 
behind  it  and  demanded  results.  Otherwise  stovepipe  issues  would  be  very 
difficult  to  overcome.  Allocation  of  resources  also  becomes  increasingly 
problematic. 

Louisville  Construction:  Yes. 

Louisville  Engineering:  Even  though  we  are  aware  that  other  districts  are 
attempting  to  cover  other  functions  in  addition  to  engineering  under  one 
certificate,  we  do  not  think  this  would  be  practical  unless  those  functions  were 
under  the  responsibility  of  the  same  division  chief  or  the  management 
representative  was  located  at  the  executive  office  level.  Even  if  this  were  the  case, 
it  would  be  imperative  that  the  assigned  management  representative  take  a  very 
active  role  in  the  development  process.  The  stovepipe  mentality  has  to  be 
eliminated. 

Portland  Engineering:  Unless  there  is  unanimity  between  the  divisions,  a  single 
division  at  a  time  is  more  manageable,  then  work  toward  adding  offices, 
ultimately  to  district  level. 
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20.  Has  the  ISO  9000  program  resulted  in  any  savings  to  date?  If  so,  how  measured? 

Savannah  Construction:  It  is  premature  to  try  to  measure  savings  as  of  this  date. 
We  implemented  our  ISO  9000  system  on  7  Oct  97  and  have  only  conducted  one 
round  of  internal  audits  to  date.  We  have  found  some  problems  that  are  being 
addressed.  This  should  lead  to  increased  efficiency  but  cannot  be  quantified  as  of 
yet. 

Louisville  Construction:  We  feel  that  for  the  first  time  we  now  have  all  that  we  do 
under  one  document.  That  is  a  benefit.  And  by  using  the  intranet,  all  ISO 
references  (ARs,  ERs,  EPs,  CEGS,  etc.)  are  linked  to  these  documents  for  ready 
reference.  This  eliminates  the  need  for  hard  copies  on  a  file  shelf  and  the  question 
of  whether  or  not  they  are  up-to-date.  The  benefits  have  not  been  measured. 

Louisville  Engineering:  We  have  not  taken  the  time  yet  to  quantify  savings,  but 
we  believe  the  process  has  increased  the  general  awareness  of  quality,  helped  tie 
down  customer  expectations,  and  facilitated  the  communication  of  these 
expectations.  We  see  more  work  coming  from  DPWs,  we  use  the  process  in  filling 
our  commitments,  and  it  serves  as  a  basis  for  making  management  decisions  on 
staff,  training,  etc.  It  has  become  a  basis  for  our  business. 

Portland  Engineering:  Incalculable  with  time  and  effort  saved.  Documentation 
initially  decreased  but  has  effectively  maintained  a  value-added  level  since  ISO 
implementation. 

21.  Do  you  have  any  performance  measures  in  place  to  measure  quality,  savings,  or 
improvements  in  efficiency  on  a  regular  basis?  If  so,  what  are  they? 

Savannah  Construction:  We  have  developed  a  set  of  audit  points  and  a  database 
to  track  the  audit  results.  With  these  two  tools,  we  can  now  do  some  statistical 
quality  control  on  how  well  we  implement  our  processes.  This  will  allow  us  to 
track  progress  by  audit  point,  by  process,  by  office,  etc.  We  never  had  this  type  of 
tool  before. 

Louisville  Construction:  Since  this  division  provides  a  service,  not  a  product,  the 
management  team  decided  to  use  the  existing  performance  indicators  required  by 
higher  authority  to  measure  quality.  Other  performance  measures  are  the  existing 
quality  assurance  plans  for  the  field  offices  and  efforts  to  seek  customer  feedback. 
Becoming  more  efficient  in  what  we  do,  through  continuous  improvement,  is  the 
savings. 

Louisville  Engineering:  We  are  not  yet  in  a  position  to  be  able  to  quantify  die 
long-term  benefits  of  being  ISO  9000  registered,  but  the  information  listed  in 
response  to  Question  14  was  our  general  assessment  as  of  early  summer  1997. 

One  additional  observation  is  that  the  Louisville  District  controllable  cost  growth 
for  the  military  construction  program  dropped  from  4.4  percent  in  FY96  to  2.1 
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percent  in  FY97.  Although  we  certainly  do  not  claim  full  credit  from  ISO  9000, 
we  do  believe  that  our  registration  efforts  contributed  to  this  drop. 

Portland  Engineering:  Command  management  review  measures  and  customer 
svuveys  are  regularly  used. 

22.  What  are  the  positive  benefits  of  managing  using  the  ISO  9000  framework? 

Savannah  Construction:  It  imposes  an  order  and  discipline  to  our  overall 
management.  The  comprehensive  audits  identify  problem  areas  that  the  system 
then  requires  be  addressed.  This  requires  analysis  of  the  problem  and  institution  of 
rewrites  to  our  procedures,  additional  training  being  provided,  etc. 

Louisville  Construction:  Some  benefits  are  mentioned  in  the  answer  to  Question 
20.  Another  benefit  is  the  synergy  we  all  felt  with  the  management  of  the  division 
pulling  together  toward  this  common  goal,  ISO  registration.  We  most  definitely 
are  working  better  as  a  team.  Once  we  obtain  registration,  we  will  no  doubt 
observe  other  benefits. 

Louisville  Engineering:  See  the  response  to  Question  14  above. 

Portlarui  Engineering:  Procedures  are  more  definitive  and  readily  understood.  K 
deviations  occur,  the  quality  manual  is  consulted  for  correct  use.  Improved 
knowledge  and  clarity  of  process,  better  customer  coordination,  consistency,  and 
mandated  management  support  and  required  proof  of  quality  control  effort. 

23.  What  were  the  negative  aspects  of  the  program? 

Savannah  Construction:  It  is  change,  and  a  lot  of  people  do  not  support  change  of 
any  type.  It  is  also  more  resource  intensive  to  implement  (but  probably  not  to 
maintain)  than  we  ever  thought  it  would  be. 

Louisville  Construction:  Cost. 

Louisville  Engineering:  Other  than  the  initial  costs,  we  cannot  identify  any  other 
negatives  aspects.  Our  primary  customer  (PPMD)  has  complained  that  we  require 
too  much  information  in  order  to  begin  initiating  action  on  their  requests.  We 
believe  the  information  we  request  is  merely  information  that  the  regulations 
require  them  to  gather  in  the  initial  phase  of  a  project,  which  typically  was  ignored 
in  the  past  and  sometimes  continues  even  now  to  be  ignored.  Our  insistence  on 
having  a  defined  scope  of  work  is  merely  requiring  them  to  fulfill  an  existing 
requirement  that  before  was  sometimes  being  ignored  in  the  name  of  providing 
quick  execution.  This  is  an  example  of  us  discovering  how  our  previous  practices 
were  less  than  satisfactory  and  fi'equently  resulted  in  a  dissatisfied  customer 
because  of  failure  to  insist  on  a  defined  and  mutually  accepted  project  scope. 
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Portland  Engineering:  None,  save  comments  such  as  “how  much  did  you  save” 
and  “what  are  the  dollar  savings  for  your  effort.”  ISO  implementation  is  a 
continuous  process,  and  long-term  effects  are  not  readily  recognized  or  discovered 
in  manifest  ways. 

24.  What  suggestions  do  you  have  for  other  districts  considering  pursuing  registration? 

Savannah  Construction:(X)  Get  buy-in  from  senior  management  up  front.  (2)  Sell 
the  benefits  to  all.  (3)  Commit  the  necessary  resources  to  get  a  system  in  place  in 
less  than  a  year.  (4)  Learn  all  that  you  can  from  the  first  districts  that  tried  this. 

Louisville  Construction:  Follow  the  footsteps  of  those  who  have  gone  before  you. 
Obtain  copies  of  their  written  quality  systems  and  rely  on  their  experiences  in 
developing  your  own  system.  Speak/meet  with  those  who  have  “been  there,  done 
that,”  and  build  on  those  experiences. 

Louisville  Engineering:  We  recommend  registration.  One  of  the  primary  benefits, 
however,  is  the  process  leading  up  to  registration  not  just  obtaining  the  certificate. 
Without  the  knowledge  and  insight  gained  by  going  through  the  process,  benefits 
would  be  greatly  reduced.  We  are  convinced  that  without  going  through  the 
growing  pains,  a  district  would  never  really  understand  its  own  processes  and 
never  become  aware  of  vulnerable  areas.  We  do  not  believe  that  a  “poor-man’ s- 
ISO”  would  work  because,  without  the  continual  incentive  of  having  to  undergo 
surveillance  audits,  any  firm  would  tend  to  slip  back  into  old  patterns  of  work  that 
do  not  meet  the  minimum  requirements  of  the  standard.  It  is  imperative  that  the 
executive  level  strongly  believes  and  supports  the  ISO  quality  program  for  it  to  be 
successful. 

Portland  Engineering:  Retain  a  good  consultant  who  understands  your  business 
operating  system  and  can  provide  a  perspective  of  ISO  requirements  with  that 
system.  Do  not  “buy”  your  registration  through  a  third-party  consultant;  it  only 
works  if  you  do  it  yourself;  you  need  to  own  the  system.  Do  it  if  you  think  you 
will  benefit  in  the  long  term! 
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